5 ms·

I so much want this to be true, I can't wait for it. Not having a layered memory architecture means no VM, everything has be rethought from the filesystem down.

I so much want this to be true, I can't wait for it. Not having a layered memory architecture means no VM, everything has be rethought from the filesystem down. On a related note: "Operating systems have not been taught what to do with all of this memory, and HP will have to get very creative." Anyone knows what this could possibly mean?

sp332 12y ago
Well, you wouldn't have to save files. You could just use a data structure in main memory because it's non-volatile. But how would you share data between programs when there's no disk? And you could execute binaries directly from storage without "loading" them into memory first. But you'd have to keep an original copy somewhere in case it modifies itself as it runs.
- markbnj 12y ago
  You'd probably still have memory partitioned into persistent areas and runtime heap of a sort, and you might very well continue to use the file system metaphor because it still makes sense to humans. Whether a file is bits on magnetic disk, in flash memory, or fast persistent RAM isn't all that meaningful to the user. As for sharing, I don't see why this would be an issue. If you have to move memory contents between machines then you still need some sort of external transport, whether it is a network, or a glowing glass cube.
  - hderms 12y ago
    Took the words right out of my mouth (better said, of course).
  - sp332 12y ago
    Filesystem semantics are a bottleneck. Fusion IO had to get rid of them when they demoed their billion-IOPS platform last year. https://news.ycombinator.com/item?id=3434711 https://news.ycombinator.com/item?id=3434711 For communicating data between programs, would you create a filesystem object that points to memory inside your program, or what?
    - DelwinAmber 12y ago
      Yes. On Linux it's /dev/shm and it is a ramdisk that you can access from multiple processes. IPC (Inter-Process Communication) is a known technology. This won't change it much if at all.
    - colanderman 12y ago
      That doesn't make any sense. Create a file, memory-map it, bam, you have RAM performance with "filesystem semantics". (There is NO overhead associated with memory-mapping a file on a ramdisk in the Linux kernel.) Probably what they're referring to is something more along the lines of replacing SCSI or NFS with a protocol that is a better match for the random access patterns used to access in-memory structures. (Hence their 64-byte I/O size; that is equal to a cache line on Intel architectures.)
      - sp332 12y ago
        Hm, how would this memristor stuff connect to the system? Will it look like a memory controller or more like PCI-e flash?
        wmf 12y ago
        It sounds like HP is pitching it as memory.
  - melloclello 12y ago
    Continuing to use the file-system metaphor (Instead of, I don't know, anything else, perhaps some kind of global namespace with built-in versioning like Urbit's) seems like lobotomising this incredible new paradigm before it is even born.
    - markbnj 12y ago
      It might be. I'm not arguing that it is the ultimate best metaphor to use, but these things do tend to be adopted organically and become very entrenched.
- bkurtz13 12y ago
  Some kind of git-like diff-based snapshot system for programs?
- hderms 12y ago
  I mean, there should still be a need for both the data and code sections of a program. Similarly, some kind of tree-like structure for naming "files" locations (for interprocess communication). I would imagine, at least in the early days of some diskless architecture, we'd have something that looks very similar to how computing is currently done but in one memory space.
- yeukhon 12y ago
  What's the issue with RAM disk to address your concern about "no disk"?
  - sp332 12y ago
    It's a lot of unnecessary copying. The data is already in RAM, but you "save" it to a different RAM location, in the RAM disk. Or if you do all your work directly on the file in the RAM disk, you have to deal with filesystem overhead constantly.
    - colanderman 12y ago
      Or if you do all your work directly on the file in the RAM disk, you have to deal with filesystem overhead constantly. Not so. Take a look at the Linux kernel implementation of memory-mapped ramdisk files. There is no "filesystem overhead". Zero. The kernel pretty much sets up the page mapping and then is hands-off. Even if your memory mapping is backed by a file on a block device, the only overhead is occasional paging to disk. This happens on the order of SECONDS… not "constantly"!
      - sp332 12y ago
        That's assuming you already have a file and know how big it should be. Or does it take the whole space for one file, and every program on the computer shares the space?
        colanderman 12y ago
        Files can be both sparsely and dynamically allocated.
    - AnthonyMouse 12y ago
      You save things to persistent storage because you want the copy. In the large majority of cases the format on the filesystem is different from the format in memory because the file format is meant to be more readable by humans or by machines with a different endianness or pointer size etc., whereas data in memory is binary and platform-specific to optimize for fast access by the local machine. The benefit of that distinction doesn't go away just because you make the persistent storage faster.
- danmaz74 12y ago
  I think there would still be a need for "files", ie, a way to store a self-contained document format in memory that is independent from the code that is working on the data at one exact moment: to allow moving documents between computers, but also to allow updating the software in a safe way (ie without losing data), to save older version of a document, but most importantly IMHO to be able to create snapshots that can be safely reloaded in case of a system crash. We're not ready to have systems that can't be rebooted.
- colanderman 12y ago
  Filesystems aren't going to disappear. Databases haven't disappeared, even though there exist 256 GB RAM monsters in data warehouses. The logical structure of a filesystem is still a useful abstraction (and has been decoupled from physical constraints for a long time now).
- colanderman 12y ago
  And you could execute binaries directly from storage without "loading" them into memory first. Linux already does this. Program code is cached copy-on-write in shared memory. The fact that, say, bash has to be loaded from disk once is pretty much irrelevant. After that, the kernel's just managing pointers.
oddevan 12y ago
Probably has to do with the way we interact with RAM (memory addresses?) vs persistent memory (file system). Can those two be merged, or is the whole system like a giant RAM disk?
colanderman 12y ago
Virtual memory doesn't exist solely to support swap files. It also allows any program to assume it's the only thing running (which simplifies assembly code) and enables memory-mapping of files and devices. (Yes, you'll still need to memory-map files when we have memristors. They still won't necessarily be contiguous on disk, and they still won't necessarily exist locally.)
enjoy-your-stay 12y ago
I guess it would be like an OS that once loads and starts, will run forever. You can put your machine to sleep which will freeze the sate, and then bring it back on again and start processing instructions from where it left off. But if that's the case then they'll have to think of ways to do e.g. OS upgrades - where you normally reboot to flush the old code out of RAM. Or how will you recover if you load a bad driver end experience a blue screen? Normally you reboot to flush the bad state and start again. Somehow I think you'll need a separate cut down immutable OS alongside the your System so that you can repair things or make updates to the main RAM/Storage space. Or at least that's how I understand it..?
- AnthonyMouse 12y ago
  I really don't understand this whole discussion about redesigning operating systems for this. Just because you have a machine that doesn't lose memory state on power failure doesn't mean you can't reboot it.
melloclello 12y ago
> Fink has assigned one team to develop the open-source Machine OS, which will assume the availability of a high-speed, constant memory store. Another team is working on a stripped-down version of Linux with similar aims; another team is working on an Android version, looking to a point at which the technology could trickle down to PCs and smartphones. I was hoping for a second there that somebody at HP had read about Loper OS and Stanislav Datsovskiy's ideas[1], or even about Symbolics LISP machines (great article on the frontpage today[2]), but nope, looks like they are just gonna repeat the same insanity of the past fifty years instead of actually doing something that will progress the field of computing. [1] http://www.loper-os.org/?p=284 http://www.loper-os.org/?p=284 [2] https://news.ycombinator.com/item?id=7878679 https://news.ycombinator.com/item?id=7878679
stevenhuang 12y ago
It means something like this has never done before, and so HP would be the first to develop the software to do this!

sp332 12y ago

Well, you wouldn't have to save files. You could just use a data structure in main memory because it's non-volatile. But how would you share data between programs when there's no disk? And you could execute binaries directly from storage without "loading" them into memory first. But you'd have to keep an original copy somewhere in case it modifies itself as it runs.

markbnj 12y ago
You'd probably still have memory partitioned into persistent areas and runtime heap of a sort, and you might very well continue to use the file system metaphor because it still makes sense to humans. Whether a file is bits on magnetic disk, in flash memory, or fast persistent RAM isn't all that meaningful to the user. As for sharing, I don't see why this would be an issue. If you have to move memory contents between machines then you still need some sort of external transport, whether it is a network, or a glowing glass cube.
- hderms 12y ago
  Took the words right out of my mouth (better said, of course).
- sp332 12y ago
  Filesystem semantics are a bottleneck. Fusion IO had to get rid of them when they demoed their billion-IOPS platform last year. https://news.ycombinator.com/item?id=3434711 https://news.ycombinator.com/item?id=3434711 For communicating data between programs, would you create a filesystem object that points to memory inside your program, or what?
  - DelwinAmber 12y ago
    Yes. On Linux it's /dev/shm and it is a ramdisk that you can access from multiple processes. IPC (Inter-Process Communication) is a known technology. This won't change it much if at all.
  - colanderman 12y ago
    That doesn't make any sense. Create a file, memory-map it, bam, you have RAM performance with "filesystem semantics". (There is NO overhead associated with memory-mapping a file on a ramdisk in the Linux kernel.) Probably what they're referring to is something more along the lines of replacing SCSI or NFS with a protocol that is a better match for the random access patterns used to access in-memory structures. (Hence their 64-byte I/O size; that is equal to a cache line on Intel architectures.)
    - sp332 12y ago
      Hm, how would this memristor stuff connect to the system? Will it look like a memory controller or more like PCI-e flash?
      - wmf 12y ago
        It sounds like HP is pitching it as memory.
- melloclello 12y ago
  Continuing to use the file-system metaphor (Instead of, I don't know, anything else, perhaps some kind of global namespace with built-in versioning like Urbit's) seems like lobotomising this incredible new paradigm before it is even born.
  - markbnj 12y ago
    It might be. I'm not arguing that it is the ultimate best metaphor to use, but these things do tend to be adopted organically and become very entrenched.
bkurtz13 12y ago
Some kind of git-like diff-based snapshot system for programs?
hderms 12y ago
I mean, there should still be a need for both the data and code sections of a program. Similarly, some kind of tree-like structure for naming "files" locations (for interprocess communication). I would imagine, at least in the early days of some diskless architecture, we'd have something that looks very similar to how computing is currently done but in one memory space.
yeukhon 12y ago
What's the issue with RAM disk to address your concern about "no disk"?
- sp332 12y ago
  It's a lot of unnecessary copying. The data is already in RAM, but you "save" it to a different RAM location, in the RAM disk. Or if you do all your work directly on the file in the RAM disk, you have to deal with filesystem overhead constantly.
  - colanderman 12y ago
    Or if you do all your work directly on the file in the RAM disk, you have to deal with filesystem overhead constantly. Not so. Take a look at the Linux kernel implementation of memory-mapped ramdisk files. There is no "filesystem overhead". Zero. The kernel pretty much sets up the page mapping and then is hands-off. Even if your memory mapping is backed by a file on a block device, the only overhead is occasional paging to disk. This happens on the order of SECONDS… not "constantly"!
    - sp332 12y ago
      That's assuming you already have a file and know how big it should be. Or does it take the whole space for one file, and every program on the computer shares the space?
      - colanderman 12y ago
        Files can be both sparsely and dynamically allocated.
  - AnthonyMouse 12y ago
    You save things to persistent storage because you want the copy. In the large majority of cases the format on the filesystem is different from the format in memory because the file format is meant to be more readable by humans or by machines with a different endianness or pointer size etc., whereas data in memory is binary and platform-specific to optimize for fast access by the local machine. The benefit of that distinction doesn't go away just because you make the persistent storage faster.
danmaz74 12y ago
I think there would still be a need for "files", ie, a way to store a self-contained document format in memory that is independent from the code that is working on the data at one exact moment: to allow moving documents between computers, but also to allow updating the software in a safe way (ie without losing data), to save older version of a document, but most importantly IMHO to be able to create snapshots that can be safely reloaded in case of a system crash. We're not ready to have systems that can't be rebooted.
colanderman 12y ago
Filesystems aren't going to disappear. Databases haven't disappeared, even though there exist 256 GB RAM monsters in data warehouses. The logical structure of a filesystem is still a useful abstraction (and has been decoupled from physical constraints for a long time now).
colanderman 12y ago
And you could execute binaries directly from storage without "loading" them into memory first. Linux already does this. Program code is cached copy-on-write in shared memory. The fact that, say, bash has to be loaded from disk once is pretty much irrelevant. After that, the kernel's just managing pointers.

oddevan 12y ago

Probably has to do with the way we interact with RAM (memory addresses?) vs persistent memory (file system). Can those two be merged, or is the whole system like a giant RAM disk?

colanderman 12y ago

Virtual memory doesn't exist solely to support swap files. It also allows any program to assume it's the only thing running (which simplifies assembly code) and enables memory-mapping of files and devices. (Yes, you'll still need to memory-map files when we have memristors. They still won't necessarily be contiguous on disk, and they still won't necessarily exist locally.)

enjoy-your-stay 12y ago

I guess it would be like an OS that once loads and starts, will run forever. You can put your machine to sleep which will freeze the sate, and then bring it back on again and start processing instructions from where it left off. But if that's the case then they'll have to think of ways to do e.g. OS upgrades - where you normally reboot to flush the old code out of RAM. Or how will you recover if you load a bad driver end experience a blue screen? Normally you reboot to flush the bad state and start again. Somehow I think you'll need a separate cut down immutable OS alongside the your System so that you can repair things or make updates to the main RAM/Storage space. Or at least that's how I understand it..?

AnthonyMouse 12y ago
I really don't understand this whole discussion about redesigning operating systems for this. Just because you have a machine that doesn't lose memory state on power failure doesn't mean you can't reboot it.

melloclello 12y ago

> Fink has assigned one team to develop the open-source Machine OS, which will assume the availability of a high-speed, constant memory store. Another team is working on a stripped-down version of Linux with similar aims; another team is working on an Android version, looking to a point at which the technology could trickle down to PCs and smartphones. I was hoping for a second there that somebody at HP had read about Loper OS and Stanislav Datsovskiy's ideas[1], or even about Symbolics LISP machines (great article on the frontpage today[2]), but nope, looks like they are just gonna repeat the same insanity of the past fifty years instead of actually doing something that will progress the field of computing. [1] http://www.loper-os.org/?p=284 http://www.loper-os.org/?p=284 [2] https://news.ycombinator.com/item?id=7878679 https://news.ycombinator.com/item?id=7878679

stevenhuang 12y ago

It means something like this has never done before, and so HP would be the first to develop the software to do this!