5 ms·
These are going to change things for a lot of reasons, not the least of which is significantly higher transaction rates in systems. In particular being past a m
by ChuckMcM 8y ago
These are going to change things for a lot of reasons, not the least of which is significantly higher transaction rates in systems. In particular being past a memory barrier means that your cache is consistent and persistent across reboots in nanoseconds rather than milliseconds or even microseconds means large transnational systems can do more operations at up to 6 orders of magnitude faster. In file systems, and especially so called 'IRON' or error corrected systems, this will significantly boost the speed of operations as more writes can be delayed safely to insure that dense spinning media is writing sequentially rather than randomly. Using the old NetApp cluster architecture I expect you could get a couple of gigabytes per second of random (from the clients) R/W performance on a very reasonable number of drives.
Of course it also makes it possible to rapidly reboot into an OS if all you need in the boot loader is to copy a gigabyte of 'ram' from one place to another and then jump to it. What that enables is powering down servers that aren't being used with the ability to power them up in milliseconds rather than seconds. That has been of Google and other cloud providers to have 'power proportionate computing' clusters.
Very cool stuff indeed.
- baq 8y agofor rebooting, you'd have to have super-fast POST and device initialization after power cycle, so i wouldn't say milliseconds.
- linsomniac 8y agoDefinitely. Most of my Dell machines take 2-5 minutes to get to the "booting OS" part of the process, but a couple of them have at times taken 3 hours. The latter is probably a firmware bug, but the best Dell ProSupport has been able to do to solve it is "flea drain" (unplug, hold the power button for 60 seconds, plug back in).
- arghwhat 8y agoDell servers are notoriously slow to boot, but this is due to the remote management subsystem being extremely slow. It's quite a pain in the arse, and one of the reasons we prefer to use Supermicro machines for development at work, using only Dell for testing.
- gascan 8y agoAt least we've been moving glacially in that direction, thanks to UEFI & faster booting. POST, device initialization, etc has begun to stick out, and is slowly being improved.
- martincmartin 8y agoHe doesn't mean rebooting will take microseconds/nanoseconds, he means the transaction commit will take that long. Before, a transaction commit meant flushing disk, at least as far as the disk controller. Now, commit = memory barrier.
- robotresearcher 8y ago"What that enables is powering down servers that aren't being used with the ability to power them up in milliseconds rather than seconds." (great-grandparent)
- ChuckMcM 8y agoActually and "restarting". Technically it isn't "booting" because the OS is already in memory and initialized, it just isn't running. Given your typical motherboard which you tell to bypass POST, it can go from power applied to memory controllers alive in just a few microseconds. The code then to go from there to running is also quite small, especially if there aren't things like graphics controllers on the PCI bus that have to be initialized first.
- namibj 8y agoAnd even those can probably be lazy-loaded, with a page fault in the IOMMU and handled as soon as the rest is done, or the rest isn't doing anything and something is thus blocking on the GPU initialisation.
- Immortalin 8y agoYou can write your own custom firmware and skip the unnecessary bits.
- bbulkow 8y agoThe aerospike demo Intel did at the event showed 17 seconds vs over 2000 seconds for a restart. The difference being rebuilding a dram index on a many many terrabyte data set. Remembering aerospike has consistency, so you can't have the index be slightly wrong.
- rasz 8y agoclose your laptop lid, wait 10 seconds, open it up, did it resume quickly?
- dylan604 8y ago2017 MacBook Pro 15". No, it doesn't. Open the lid, time passes before screen is no longer black. Even longer amount of time before the TouchID figures out my finger is the accepted finger. So, when did it resume? When the screen lit up, or when I could actually use it? Obviously, Apple's TouchID is preventing me from resuming, but to the user, that's all they care about.
- rasz 8y agoYou made the choice buying Apple.
- PaulHoule 8y agoAt the current rating of 30 drive writes per day if you try to use it the way people use RAM now it will maybe last a week if you're lucky.
- ChuckMcM 8y agoIt is important to remember that this isn't FLASH. It is more like FRAM with its non-volatility. When Intel and Micron announced it three years ago[1] they made some pretty bold claims, like 1000x faster than NAND, unlimited write endurance, Etc. For a long time it was just vaporware and then it came out as "NVME" type SSDs and now they have figured out how to put it right on the frontside bus with the memory controller. [1] https://newsroom.intel.com/news-releases/intel-and-micron-produce-breakthrough-memory-technology/ https://newsroom.intel.com/news-releases/intel-and-micron-pr...
- adrianmonk 8y ago> if all you need in the boot loader is to copy a gigabyte of 'ram' from one place to another and then jump to it I suppose you don't even have to copy it before you jump to it. You could map the pages and then lazily copy when you get page faults. > powering down servers that aren't being used with the ability to power them up in milliseconds Still, if this is your goal, can't you get 90% of the way there by keeping some DRAM powered up while everything else is off? Refreshing DRAM requires some power, but not that much from what I understand (as a software guy).
- toxik 8y agoNovice in these contexts but I think it would be less of a saving as DRAM must be refreshed, at a pretty high clockrate too, typically.
- arghwhat 8y agoDRAM has a self-refresh mode powered by an internal clock that you use for low-power standby, so this is not a problem. The DRAM just need to be powered.
- wtallis 8y ago> I suppose you don't even have to copy it before you jump to it. You could map the pages and then lazily copy when you get page faults. You could also just map it and not even fault on access. For data that's mostly read and seldom written, and not too much of a hot spot, there would be no reason to move it to DRAM. Depending on the read latency of these DIMMs, that might include a lot of executable code.
- ChuckMcM 8y agoOn keeping DRAM alive, at NetApp where the company built custom motherboards specifically for filers, this continued to be an issue for the memory controllers in the processor chips. Neither AMD nor Intel had any support for staying clear of the DRAM configuration on power up of their server chips. There was an interesting config in one of their laptop chips but no ECC on that chip so it was a non starter. At Google the motherboards were less customized and even less likely to incur a Google specific cost of something like holding up memory. The benefit of this coming from Intel is that they do the work to qualify it and everyone else gets to reap the benefits. The memory bus on these processors is 3 - 30 GBs so copying a full gig would be a 3 - 30mS sort of deal.A pretty short start time. And yes, you tell the BIOS to skip POST (its a common start up option in even off the shelf BIOS packages these days) in order to get a faster boot time.
- arghwhat 8y agoOf course, it comes at a cost: These dimms have several orders of magnitude higher latency and a fraction of the throughput of DRAM. Once you introduce volatile memory as highspeed buffer, the coolness factor drops quite a bit as you end up in the same volatile/non-volatile tiered storage split that we have always had. It's just back on a parrallel bus like in the good ol' days of PATA.
- tigershark 8y agoHow do you know? Other comments have pointed out that Intel was aiming for Optane DIMMs with a read latency half of the current RAM.
- arghwhat 8y agoI appear to have exaggerated slightly when I said several orders of magnitude, but the original 3D XPoint latency numbers put it at about 25 times higher latencies than contemporary DRAM (~7000ns vs. ~200ns). Likewise, quad-channel DDR4 setups can push around 60GB/s, while Optane PCIe SSD's have only been able to push around 2GB/s. The PCIe x4 interfaces that they use should have been able to push 4GB/s, and if they started hitting a wall there, they could just have used a full PCIe x16 interface which could push around 15GB/s. This indicates that the throughput was not interface bound. Nothing seems to signal that Intel have have anything up their sleeve to provide the 25x latency improvement and 30x throughput improvement needed to just be even with DRAM, although placing the chips on the memory bus will of course provide some speedup.
- ChuckMcM 8y agoNote that transactional costs on PCIe are much higher than they are for memory, and you are going through a disk driver (if you're and SSD, you can avoid that using the NVME architecture extensions) There is no driver needed for non-volatile storage on the memory bus. You just access it like memory. I would not be surprised if early users of these devices were seeing 10GBs+ of data throughput.