15 ms·
Intel’s first Optane SSD: 375GB that you can also use as RAM
- piinbinary 10y agoWith drives like this, the approach of "throw more hardware at it" continues working for databases to the point where most database loads in the world can be handled on a single machine.
- rosser 10y agoAs someone who's worked with PostgreSQL for a decade now, I've very definitely seen cases where memory bandwidth is the bottleneck, not disk bandwidth. I'm pretty sure the IO subsystem would still be a narrower pipe than main memory, even if you stuck Optane devices in every available connector at the same time and striped across them. EDIT: Don't mistake me, I'm very excited about the potential of Optane devices in the workloads my databases handle (though, since my postgres machines are all in the cloud right now, that remains a purely theoretical question). It's just not a panacea. Then again, nothing is.
- zzzcpan 10y ago"where most database loads in the world can be handled on a single machine" I don't think it works like that. There is always some point when at certain number of users avoiding downtime becomes more important, than handling a load and hardware performance stops matter as much.
- Shivetya 10y agohigh availability systems can counter this and if properly implemented you could share certain workloads with both the primary and HA systems; basically anything which doesn't change data. the its memory and storage would be a boon to dynamic indexing and common table expressions. maybe this new technology will bring single level storage as IBM has employed on some machines to the public. Where all storage is treated as one resource and only the machine knows the difference.
- jrockway 10y agoScaling beyond what fits inside one computer is a reasonable concern that this addresses, but ultimately that one computer can be sucked up by a tornado at an inconvenient time, so distributed systems will always be necessary for availability and durability.
- yjftsjthsd-h 10y agoResiliency, yes, but maybe not performance. Still a great win :)
- peterwwillis 10y agoThere are faster drives. Load is just one factor in database design. For example, if you want fast query returns, the speed of light will limit you if you're using one machine and your query comes from around the world. Then there's the whole high availability thing. One single machine just doesn't fit every scenario.
- kartan 10y agoThe more powerful the computers, the more data we want to aggregate and process. All that tracking that companies are gathering has to be used somewhere. Also SaaS means that one provider holds data of thousands of companies. Salesforce, for example, has more than 100,000 customers. So the scale is still too big for one machine (or two or three for redundancy).
- brianwawok 10y agoSure but Salesforce has natural sharding. Customers never need to see each other's data. So can a single DB hold the largest customer? That is more the question here...
- snuxoll 10y ago> Customers never need to see each other's data. Correct and incorrect at the same time. Separate orgs may not need direct access to each others rows in the multi-tenant database, but there are plenty of use cases where data does need to be shared between them (companies with multiple orgs, like ours - business parters, etc) and Salesforce has tooling to handle this (Salesforce to Salesforce, Lightning Connect).
- gbrown_ 10y agoIt feels like Intel are putting this out in lieu of actual hardware. As always remain skeptical until real silicon appears on the market.
- sciurus 10y agoIsn't the article about real silicon appearing on the market? "Initial limited availability starts today, for $1520"
- gbrown_ 10y agoThe "limited availability" is almost certainly for those with per-arranged deals with Intel and 2H is almost always Q4. It's not to say this isn't an interesting product but I've a feeling Optane won't live up to the hype. But I'm reserving judgment until it gets into the wider market/ has more real world exposure.
- georgeburdell 10y agoIf by market you mean someone not Intel. They probably sold all their capacity for the year to certain key customers, and possibly even with the parts not meeting spec (some companies won't care if they still see a boost), and the small guy will not see it until next year.
- olavgg 10y agoI wonder what the performance of these are with PostgreSQL's pg_test_fsync, which is one of the proper tools to benchmark a SSD. I get 4000 iops with my Intel 320 SSD and 9000 iops with Intel S3700. For comparison, Intel 600p maxes around 1500 iops, and Samsung 750 Evo at 400 iops.
- snaky 10y ago"Why IOPS Suck and Everything You Know About Them is Probably Wrong!" - https://www.youtube.com/watch?v=cEb270L5Q1Y https://www.youtube.com/watch?v=cEb270L5Q1Y
- deleted 10y ago[deleted]
- sologoub 10y agoAt previous employer, we built a system using Druid as the primary store of reporting data. The setup worked amazingly well with the size/cardinality of the data we had, but was constantly bottlenecked at paging segments in and out of RAM. Economically, we just couldn't justify a system with RAM big enough to hold the primary dataset. As the result, we had to prioritize data aggressively, focusing on the more recent transactions and locating them on the few servers with very high RAM that we did have. Historic data segments had to go through a lot of paging in/out of RAM. User experience on YTD (year-to-date) or YOY (year-over-year) reports really suffered as the result. I don't have access to the original planning calculations anymore, but 375GB at $1520 would definitely have been a game changer in terms of performance/$, and I suspect be good enough to make the end user feel like the entire dataset was in memory.
- Redsquare 10y agoWould you choose to run with druid again?
- sologoub 10y agoFor that use case, absolutely! We made do with the version that could not even support label appends (limited joins). The current version would allow us a lot fewer workarounds. The probabilistic hyperloglog data type is also a game changer compared to say redshift, but again it's only viable if you are dealing with counting (estimating) unique entities across billions of rows and super-wide dimension sets. If you are doing a general purpose analytics store, Redshift is hard to beat because of reliability and ease of implementation. Druid is a purpose-built race car. Redshift is a good cross-over - far less headache and can do almost any job good enough, but you won't have the tuning or performance (when tuned right) at scale. Although, I'm continuously impressed with what redshift actually can do, dispite the humble feature set. Druid's main weakness is lack of SQL support, so it's not a great analyst datastore. You pretty much have to wrap it into a reporting app.
- otterley 10y agoWhat do you think of ClickHouse vis-a-vis Druid and Redshift?
- ChuckMcM 10y agoYay! Nice to see these things becoming more real. The choice of U.2 is interesting, it might force wider adoption of that form factor. This is definitely going to change the way you build your computational fabric. Putting that much data that close to the CPU (closeness here measured in nS) makes for some really interesting database and distributed data applications. For example, a common limitation in MMO infrastructure is managing everyone getting loot and changing state (appearance, stats, etc). The faster you can do those updates consistently the more people you can have in a (virtual) room at the same time.
- wtallis 10y agoThe U.2 form factor has had no trouble catching on in the datacenter, which is where these first Optane SSDs are intended to be used. Intel is also offering standard half-height half-length add-in cards, and M.2 is simply too small to accommodate much 3D XPoint memory.
- xt00 10y agoThe smart thing that Intel is doing is making stuff that they know big cloud providers like AWS etc. will pay crazy amounts for and buy in huge volumes. The "use it as RAM" is incredibly valuable -- especially to bring down costs for databases. For example, running a database with an allocated 32GB of RAM is pretty expensive per month.. And if somebody like AWS sold a cheaper DB instance version that ran from this drive as its memory (or was smart paged), then that could bring down the cost of allocating huge databases to memory with a performance hit that many people would be willing to take to save the money.
- tankenmate 10y ago64 bit machines (given the right controllers etc) should be able to map the whole 375GB / 750GB directly into the memory space. You'd almost certainly need a kernel driver, similar to the way a graphics card is mapped into the address space but isn't treated as regular RAM. With the right driver you could just mmap() the address space.
- wtallis 10y agoIntel's strategy here is actually to continue using the standard NVMe interface for Optane SSDs, and to offer a hypervisor that does the job of presenting to the guest OS a pool of memory that is backed by a combination of DRAM and 3D XPoint.
- ccleve 10y agoThis is a big deal. We've always made a distinction between memory and disk. Much of computer science is about algorithms that recognize the difference between slow, persistent disk and fast, volatile RAM and account for it somehow. What happens if the distinction goes away? What if all data is persistent? What if we can perform calculations directly on persisted data without pulling it into RAM first? My guess is that we'll start writing software very differently. It's hard for me to predict how, though.
- hendzen 10y agoCMU has been conducting research in this area for a few years now. http://www.pdl.cmu.edu/NVM/index.shtml http://www.pdl.cmu.edu/NVM/index.shtml
- e12e 10y agoDifferently is right. Every buffer over/underrun can now potentially overwrite your "persisted" data - now we're all writing file/storage/db systems .... ;-) It'll be wonderful if we actually can get some simplifications in our byzantine architectures, although - I wonder if it doesn't all boil down to our "RAM" (fastest storage) becoming "smarter" - as CPUs manage the three levels of cache - and we "only" loose the "slow disk/ssd" on the far end. CPU cache becomes the new RAM, and cache management microcode becomes our new memory allocators....?
- Negitivefrags 10y agoA buffer overrun can't overwrite random memory that isn't mapped into your process. We solved this issue with virtual memory like 35 years ago.
- analognoise 10y agoSo you're saying there will be a web assembly version in a few years that will claim to be a huge advancement?
- 10y ago
- jpalomaki 10y agoSomebody linked an "Intel to mislead press on Xpoint next week" article from SemiAccurate to another thread on the same topic. Interesting read and adds some context to the announcement. http://semiaccurate.com/2017/03/10/intel-mislead-press-xpoint-next-week/ http://semiaccurate.com/2017/03/10/intel-mislead-press-xpoin...
- grizzles 10y agoI'm skeptical. It must be hard for a once innovator like Intel to revert to the mean.
- rayiner 10y agoBy "broken" SemiAccurate seems to mean "only twice as fast, 10x lower latency, and 3x the endurance of NAND." So who is misleading whom?
- Lazare 10y agoIf you read the details of the article, there's some pretty open questions about whether Xpoint is 3x the endurance of NAND, or even higher duration than NAND. And the latency improvement seems to be 2x, not the 10x you claim, or the 1000x Intel originally claimed. (Intel's own numbers compare 20ms for flash versus 10ms for XPoint...) Which isn't nothing, to be sure, but...
- Dylan16807 10y agoWell the Ars article says they're still promising 1000x lower latency so that seems pretty misleading. And that 3x endurance is vs. MLC flash; it still handily loses to SLC. The SemiAccurate article looks very fair to me.
- RachelF 10y agoProbably just marketing hype. Some numbers: Modern DDR4 RAM quad channel: 50,000MB/s Optane SSD: 2,000MB/s Also nothing on the PCIe bus is going to have a latency of RAM to the OS, unless mapped as RAM.
- frozenport 10y agoI would opt for a more conventional solution, you can get 256GB of ram for under $2k, and then enable write caching.
- comboy 10y ago/dev/null is also pretty fast But for use cases where data matters, fsync seems like a reasonable idea.
- ori_b 10y agoYou can also attach batteries to the RAM. This is already a solution people ship in production.
- comboy 10y agoAny references? I tried to find something but I failed. I'm assuming we are talking about something different than UPS + going to sleep? It seems very tricky to just keep RAM powered.
- wmf 10y agoIntel has a reference design for just keeping RAM powered but it's only used in storage controllers, not in any servers you can buy.
- pkaye 10y agoThey are called NVDIMMs. One example http://www.vikingtechnology.com/arxcis-nv http://www.vikingtechnology.com/arxcis-nv I think some of them are protected by both a battery and flash memory for long term backup.
- dictum 10y agoStorage Search is the best rabbit hole on the subject of RAM SSD that comes to mind. http://www.storagesearch.com/ssd-ram.html http://www.storagesearch.com/ssd-ram.html I mean, how else would you get 80 MB/sec? http://web.archive.org/web/20010613164621/http://www.mncurtis.com/curtis/CurtisProducts/clipii.pdf http://web.archive.org/web/20010613164621/http://www.mncurti...
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- floatboth 10y agoMeanwhile, the Raspberry Pi is using a garbage microSD controller that corrupts cards… the divide between cheap and high end stuff is just mindblowing these days. Also, normal (NAND) NVMe M.2 SSDs are still TWICE as expensive as good old SATA ones, at least in my country… And they want to push Optane into the consumer space later this year… who even needs that much performance at home?
- adventured 10y ago> who even needs that much performance at home In the next ten years (closer to ten than not) we'll need it for massive and or hyper intricate 4k VR worlds, the assets of which you won't want to download every time you load up the world/s. You'll want to hold as many of the assets locally in something extremely fast, if not ram then the next best thing. That is until we commonly have 10gbps plus to the home.
- floatboth 10y agoRAM and VRAM these days are huge, and SATA is definitely fast enough to stream assets from. Heck, I load most of my games from an old 2.5" spinning HDD that I pulled from a dead laptop. I think more improvement is needed on the VRAM side than on the disk side, so HBM2 hype > Optane hype :)
- digi_owl 10y agoNothing new. Just look at a PDP-11 tape drive vs the cassette used with early micros.
- nimos 10y agoSeems like this would be ideal in cases where you are waiting for file sync in multiple locations which I assume a lot of banks/corporations do. Interesting it seems to be marketed as cheaper memory. You'd think at first they'd try and rip super high margins out of banks/corps by selling it as "persistent" memory. Although I guess if your waiting for file writes in multiple locations the network overhead makes the actual write sort of irrelevant.....
- faragon 10y agoSo finaly we're going to get from Intel what HP promised years ago as "memristors".
- petercooper 10y agoMemristors theoretically go further than mere storage. They enable the idea of data storage and processors eventually becoming the same thing. I hope they continue with the work because some of what I was reading about it was fascinating - rather than have caches, you could, in theory, write temporary routines that ran in physical locality to the data they had to process.
- Dylan16807 10y agoYou can already do that, it's just that dumb storage can be packed more densely than smart storage. Memristors can make things more dense, but that applies to both options.
- SergeAx 10y agoI wonder why no one mentioned application of this technology in mobile phones. Most obvious case: it will be posible to bring entire system from hybernate state while user pulling phone from the pocket and pressing "awake" button. Power and computing cost of hybernating/restoring system would be slightly north of zero, which leads to dramatic increasing of battery life. Not to mention size factor and lower power consumption of chip itself.
- 0x0 10y agoWhat's the endurance like if you actually use it as ram? How many times can you do a "label1: inc [rax]; jmp label1" loop having rax pointing to a particular byte address on the SSD? (With GHz CPUs, wouldn't that mean giga-writes per second? Isn't NAND rated for 10k-100k writes total, and if this is rated for 1000x more than that, you'd still hit a 10m-100m total writes in like a second?)
- gjm11 10y agoOriginally Intel claimed that this new technology would offer 1000x shorter latencies and 1000x better endurance than NAND flash, and 10x better density than DRAM. The figures they're quoting now are more like 10x shorter latencies and 3x better endurance (compared with flash), and 2.5x better density (compared with RAM). The article linked here says "3D XPoint has about one thousandth the latency of NAND flash" but I don't see any actual evidence for that. The paragraph that says it is followed by a link to actual specs for a "3D XPoint" device, saying: "the Intel flash SSD has a 20-microsecond latency for any read or write operation, whereas the 3D XPoint drive cuts this to below 10 microseconds." which sounds to me more like a 2x latency improvement than a 1000x improvement. So I ask the following extremely cynical question. Is there any evidence available to us that's inconsistent with the hypothesis that actually there is no genuinely new technology in Optane? In other words, have they demonstrated anything that couldn't be achieved by taking existing flash technology and, say, adding some redundancy and a lot more DRAM cache to it? [EDITED to add:] I am hoping the answer to my question is yes: I'd love to see genuine technological progress in this area. And it genuinely is a question, not an accusation; I have no sort of inside knowledge here.
- mtdewcmu 10y agoThe key here seems to be that performance does not drop off very much under load. DRAM cache hides latency, but only up to a point. Hiding latency is like a magic trick, and you can only hide so much before the trick starts to break down.
- wmf 10y agoThe performance consistency that Intel claims cannot be provided by caching. The characteristics of XPoint sound vaguely similar to SLC NOR flash, but it seems like Intel would be opening themselves up to liability by outright lying.
- gjm11 10y agoGood to know. Have they actually said anything about Optane that would be an outright lie if Optane devices were using already-existing hardware technology?
- dgudkov 10y agoRamdisks are back. This time persistent.
- penagwin 10y agoI was under the impression that this is the opposite. It's a 'disk' (ram disks are just storage that can store files, so I'm pretending disk means it stores files) like a SSD that's fast enough to act like ram?
- mrfusion 10y agoIs this based on memristors? That's pretty amazing. I thought hp owned that.
- aikorevs 10y agoTried to look up modern memory latency numbers but could not find. Because "numbers every programmer should know" are about 10 years old as I understand.
- Animats 10y agoThe big limitation of this device is wear. "Optane SSDs can safely be written 30 times per day", says the article. That implies a need for wear monitoring and leveling. Although you can modify one byte at a time, the need to monitor wear implies that just memory-mapping the thing isn't going to work. Wear management could be moved to hardware, though, using a special MMU/wear monitor/remapping device. If you're using this thing as a level of memory below DRAM, viewing DRAM as another level of cache, something like that would be necessary. That's one application. This device would make a good key/value store. MongoDB in hardware. Then you don't care where the data is physically placed, and it can be moved for wear management.
- pilif 10y agoFrom the article: > This gives the drives much greater endurance than NAND of a comparable density, with Intel saying that Optane SSDs can safely be written 30 times per day, compared to a typical 0.5-10 whole drive writes per day. I understand this as: to get normal length usage of traditional SSDs you can get away with writing enough data to completely rewrite the device 0.5 to 10 times, whereas with the new device, it's 30 times. The new technology is 3 times better than SSDs with regards to write durability. I think the article would be better worded if it didn't leave out the most important number: 10 rewrites per day for how long before either the SSD or the new thing fails?
- deleted 10y ago[deleted]
- viraptor 10y agoIt doesn't say that the wear leveling is not present inside the box already. It could mean rewriting the whole storage 30 times a day effectively.
- Animats 10y agoIf writes can be done at the byte level, the bookkeeping and indirection for byte level wear leveling would take more space than the data. If wear leveling is required, it has to be done on blocks of some reasonable size. They might be smaller than for flash, though.
- willdehaan 10y agoIt would be useful to move some SQLd indices to an Optane drive with DB rows on arrays of m.2 flash drives. SQL consolidation/redundancy over network out of band, slowly. This to scale somewhat cheap OSS SQL clusters. This starts to make sense in a single server node with enough 4 channel pcie interfaces. AMD Naples?