6 ms·

HP is already working on this with what they're calling "The Machine". They're writing a new OS around the new storage technologies with the understanding that

by luma 12y ago

HP is already working on this with what they're calling "The Machine". They're writing a new OS around the new storage technologies with the understanding that it might not make sense to separate RAM from disk pretty soon here. They've also suggested that they'll be porting versions of Linux and Android to this platform. Of course, all of this is pretty marketing heavy and not a lot of details have been released, so take this with a huge grain of salt. HP's CTO wrote a fluff piece on flattening the storage hierarchy here: http://www8.hp.com/hpnext/posts/end-necessary-evil-collapsing-memory-hierarchy http://www8.hp.com/hpnext/posts/end-necessary-evil-collapsin...

justincormack 12y ago
I am not sure it is even a real project. I tried to find anyone working on it and failed. Nice story though.
- innguest 12y ago
  It must be related to their work on memristors which is why they should probably keep it a secret for a while.
- andyjohnson0 12y ago
  Ars [1] and Businessweek [2][3] published articles about it recently. [1] http://arstechnica.com/information-technology/2014/06/hp-labs-machine-dissolves-the-difference-between-disk-and-memory/ http://arstechnica.com/information-technology/2014/06/hp-lab... [2] http://www.businessweek.com/printer/articles/206401-with-the-machine-hp-may-have-invented-a-new-kind-of-computer http://www.businessweek.com/printer/articles/206401-with-the... [3] https://news.ycombinator.com/item?id=7878949 https://news.ycombinator.com/item?id=7878949
- JibberMeTimbers 12y ago
  Check out stuff from 'Discover 2014'. Discover is HP's tech showcase event. I linked a video below from Day 2 of this years showcase. About halfway through, Martin Fink goes into a bit on how it's supposed to work. https://www.youtube.com/watch?v=Gxn5ru7klUQ https://www.youtube.com/watch?v=Gxn5ru7klUQ
- pbhjpbhj 12y ago
  >I tried to find anyone working on it and failed. // Inventors of anything submitted in patent applications should be named on the application. That is a potential route to finding who in a corp actually invented stuff.
- dgacmu 12y ago
  I know several researchers there who're working on it. It's for real. It's risky and ambitious, but it's a real, serious project (with tons of sub-research-projects, as you might imagine for something that requires multiple hardware advances and software advances to achieve.) (source: I'm a cs professor at carnegie mellon, and I was just chatting with some of those researchers at OSDI.)
  - justincormack 12y ago
    Cool. I wanted to find someone to talk at https://operatingsystems.io/ https://operatingsystems.io/ about the OS bits (obviously there are lots of bits to it) - its a bit late now for 2 weeks time but if you know anyone appropriate put then in touch for next time... email in profile.
- mfisher87 12y ago
  I know someone at HP who recently got a job on this project. So... it's not a lie?
  - justincormack 12y ago
    Cool, put them in touch (see post above).
  - justincormack 12y ago
    Cool, put them in touch (see post above).
- jacquesm 12y ago
  It's very much real but I'm not sure if I agree with the basic premise. 'Regular' operating systems have gone through a whole slew of storage changes relatively unscathed, what it is about faster persistent storage that you could not encapsulate the same way we did with memory mapped files I do not understand. Obviously HP is far from stupid (or at least, its scientists are) but for now I'm on the fence if this is going to be a true revolution or just the needle moving a bit further to the right. Real revolutions in OS tech have for the last decades come from the keyboards of large institutional users and from hordes of smaller ones. Even IBM has pretty much acknowledged that. So we'll see if HP will be able to keep the door locked to someone doing a linux device driver for their new hot hardware or to add a few calls to the kernel that enable an application to use this tech. The same has happened to infiniband (fast networking for clusters and storage) and tons of other esoteric hardware including shared memory between physically disjoint machines (DSM).
  - solarmist 12y ago
    You're right this isn't really a revolution in computing, but a unification. The hardware will be revolutionary, but the changes at the OS level will probably not be. Still they're merging entire sub-systems that have existed independently for decades and someone has to actually make all the changes to support permanent in-place execution of code and the like that's what I mostly got out of the OS porting news.
  - vidarh 12y ago
    There are certainly areas that might change. E.g. if HP's memristor research pays off, with true random access, consider that solutions like Redis suddenly can be written to provide persistence without having to write a log to disk or write dump the data. And e.g. consider the changes to RDBMS's if you can do guaranteed committed writes of individual bytes. A huge amount of the work on RDBMS's have been around table types that are optimised for per-block read/writes, where the design tradeoffs might be very different if you can do "cheap" random access (though caches will probably still justify some degree of block-wise access). But with respect to operating systems, I agree that it will take a long time to see any dramatic changes. The current mechanisms would still work: Open a file that maps to a memory region instead of a drive; mmap() it. Your app would just want to know whether or not the file content is instantly persistent on write or not. I think HP's starting point, though, is that we don't really know. Both in terms of hardware and software. Take this DIMM drive, for example. If you look at current server motherboards, a lot of servers has no need for PCIe slots other than for SSDs. A lot of them will not need drive slots at all if you can stuff that much storage on DIMM slots. Suddenly it becomes viable to put really high performance systems in half depth 1U form factors. For a lot of our larger servers, 50% or more of the server volume is air because current designs are so geared towards front loaded hot-swappable drives. Maybe 75% of the weight of the servers are due to drive bays and larger boxes to accommodate that front-loading. That's a lot of wasted metal, and a lot of wasted rack space. And a lot of wasted electronics for SATA interfaces that could sometime become obsolete.
digi_owl 12y ago
I seem to recall that early Palm devices used run-in-place, but i don't recall if they were able to maintain state if there was a power loss.
- zelos 12y ago
  The early ones had a capacitor that held enough charge to allow you to change the batteries without losing state.
  - dspillett 12y ago
    The Psion hand-helds (at least the 3 and 5 eries) did this too. They were powered from a pair of AA batteries and no state was lost while changing them, though IIRC it was a small button cell that gave power to keep the memory refresh going while there was no power from the AAs or the external power adaptor, not a capacitor.
    - jon-wood 12y ago
      Yeah, and that button cell didn't last as long as you might think - I remember many an occasion when I'd let mine run flat, and then leave it just a little bit too long before charging it back up again.

by luma 12y ago

justincormack 12y ago

I am not sure it is even a real project. I tried to find anyone working on it and failed. Nice story though.

innguest 12y ago
It must be related to their work on memristors which is why they should probably keep it a secret for a while.
andyjohnson0 12y ago
Ars [1] and Businessweek [2][3] published articles about it recently. [1] http://arstechnica.com/information-technology/2014/06/hp-labs-machine-dissolves-the-difference-between-disk-and-memory/ http://arstechnica.com/information-technology/2014/06/hp-lab... [2] http://www.businessweek.com/printer/articles/206401-with-the-machine-hp-may-have-invented-a-new-kind-of-computer http://www.businessweek.com/printer/articles/206401-with-the... [3] https://news.ycombinator.com/item?id=7878949 https://news.ycombinator.com/item?id=7878949
JibberMeTimbers 12y ago
Check out stuff from 'Discover 2014'. Discover is HP's tech showcase event. I linked a video below from Day 2 of this years showcase. About halfway through, Martin Fink goes into a bit on how it's supposed to work. https://www.youtube.com/watch?v=Gxn5ru7klUQ https://www.youtube.com/watch?v=Gxn5ru7klUQ
pbhjpbhj 12y ago
>I tried to find anyone working on it and failed. // Inventors of anything submitted in patent applications should be named on the application. That is a potential route to finding who in a corp actually invented stuff.
dgacmu 12y ago
I know several researchers there who're working on it. It's for real. It's risky and ambitious, but it's a real, serious project (with tons of sub-research-projects, as you might imagine for something that requires multiple hardware advances and software advances to achieve.) (source: I'm a cs professor at carnegie mellon, and I was just chatting with some of those researchers at OSDI.)
- justincormack 12y ago
  Cool. I wanted to find someone to talk at https://operatingsystems.io/ https://operatingsystems.io/ about the OS bits (obviously there are lots of bits to it) - its a bit late now for 2 weeks time but if you know anyone appropriate put then in touch for next time... email in profile.
mfisher87 12y ago
I know someone at HP who recently got a job on this project. So... it's not a lie?
- justincormack 12y ago
  Cool, put them in touch (see post above).
- justincormack 12y ago
  Cool, put them in touch (see post above).
jacquesm 12y ago
It's very much real but I'm not sure if I agree with the basic premise. 'Regular' operating systems have gone through a whole slew of storage changes relatively unscathed, what it is about faster persistent storage that you could not encapsulate the same way we did with memory mapped files I do not understand. Obviously HP is far from stupid (or at least, its scientists are) but for now I'm on the fence if this is going to be a true revolution or just the needle moving a bit further to the right. Real revolutions in OS tech have for the last decades come from the keyboards of large institutional users and from hordes of smaller ones. Even IBM has pretty much acknowledged that. So we'll see if HP will be able to keep the door locked to someone doing a linux device driver for their new hot hardware or to add a few calls to the kernel that enable an application to use this tech. The same has happened to infiniband (fast networking for clusters and storage) and tons of other esoteric hardware including shared memory between physically disjoint machines (DSM).
- solarmist 12y ago
  You're right this isn't really a revolution in computing, but a unification. The hardware will be revolutionary, but the changes at the OS level will probably not be. Still they're merging entire sub-systems that have existed independently for decades and someone has to actually make all the changes to support permanent in-place execution of code and the like that's what I mostly got out of the OS porting news.
- vidarh 12y ago
  There are certainly areas that might change. E.g. if HP's memristor research pays off, with true random access, consider that solutions like Redis suddenly can be written to provide persistence without having to write a log to disk or write dump the data. And e.g. consider the changes to RDBMS's if you can do guaranteed committed writes of individual bytes. A huge amount of the work on RDBMS's have been around table types that are optimised for per-block read/writes, where the design tradeoffs might be very different if you can do "cheap" random access (though caches will probably still justify some degree of block-wise access). But with respect to operating systems, I agree that it will take a long time to see any dramatic changes. The current mechanisms would still work: Open a file that maps to a memory region instead of a drive; mmap() it. Your app would just want to know whether or not the file content is instantly persistent on write or not. I think HP's starting point, though, is that we don't really know. Both in terms of hardware and software. Take this DIMM drive, for example. If you look at current server motherboards, a lot of servers has no need for PCIe slots other than for SSDs. A lot of them will not need drive slots at all if you can stuff that much storage on DIMM slots. Suddenly it becomes viable to put really high performance systems in half depth 1U form factors. For a lot of our larger servers, 50% or more of the server volume is air because current designs are so geared towards front loaded hot-swappable drives. Maybe 75% of the weight of the servers are due to drive bays and larger boxes to accommodate that front-loading. That's a lot of wasted metal, and a lot of wasted rack space. And a lot of wasted electronics for SATA interfaces that could sometime become obsolete.

digi_owl 12y ago

I seem to recall that early Palm devices used run-in-place, but i don't recall if they were able to maintain state if there was a power loss.

zelos 12y ago
The early ones had a capacitor that held enough charge to allow you to change the batteries without losing state.
- dspillett 12y ago
  The Psion hand-helds (at least the 3 and 5 eries) did this too. They were powered from a pair of AA batteries and no state was lost while changing them, though IIRC it was a small button cell that gave power to keep the memory refresh going while there was no power from the AAs or the external power adaptor, not a capacitor.
  - jon-wood 12y ago
    Yeah, and that button cell didn't last as long as you might think - I remember many an occasion when I'd let mine run flat, and then leave it just a little bit too long before charging it back up again.