24 ms·

Aerospike CTO here. We did a lot of testing with this drive, and the interesting bit is that performance doesn't degrade under write pressure. With a NAND drive

Aerospike CTO here. We did a lot of testing with this drive, and the interesting bit is that performance doesn't degrade under write pressure. With a NAND drive, when you push writes high ( usually needed in front-edge / microservice apps ), the read latencies take a real hit, and often get into millisecond-average at the device. Optane simply doesn't behave that way. All of that means you have to code to it differently. It isn't DRAM, it's not Flash. We've done the work to integrate and test at Aerospike ( semi-plug sorry ), so the system works. The XPoint tech gets really interesting later in the year, when three things happen: the higher density drives show up, Dell ships their high-NVMe chassis, and more clouds support it. Regarding cloud - IBM's BlueMix has announced Optane, and the other providers have a wide variety of plans. I can't comment more. Finally, Intel has been clear about putting this kind of tech on the memory bus, and that really opens the doors to some interesting choices, some data structure changes. That requires new coding, we're on it. Here's an interesting ComputerWorld article about our experience with Optane: http://www.computerworld.com/article/3188338/data-storage/what-aerospike-learned-from-testing-intels-superfast-optane-ssds.html http://www.computerworld.com/article/3188338/data-storage/wh...

bch 9y ago
> All of that means you have to code to it differently. > That requires new coding, we're on it. Can you give a rough example to guide my thinking? I understand OSes are considering ways that this has both traditional disk and RAM properties, and are sorting out storage subsystems of drivers, but I assume you're talking about something closer to user-level structures and algorithms?
- wtallis 9y ago
  Databases have traditionally gone to great lengths to bypass whatever heuristics the operating system is using for I/O scheduling and caching so that they can implement their own optimizations while maintaining their data consistency guarantees. Any changes to OS behavior you can imagine are also potentially applicable to databases that take this approach.
- bbulkow 9y ago
  Yes, I think there are clear user-level approaches you can ( and must ) take. There are some interesting talks out there about NAND, how it works, and how to optimize - I saw something here on HN a few days ago about writing your own time-series database, which got a variety of the facts wrong but was an example of how to choose data structures that are NAND-reasonable. You can look up some of my YouTube talks and slideshare, for example - I've been talking this for a while. At a high level, NAND has more IOPs than god, because they don't seek. An old enterprise spindle littlerally does 200 to 250 seeks per second. And Flash can read from 500,000 different random locations per second. That's so far apart that different user level approaches are called for. In terms of XPoint, let me give you one detail. What does a "commit" look like in XPoint? What do the different kinds of memory barriers look like? What's the best way to validate this kind of persistence on restart, which you don't have to do with DRAM? Does that change your "malloc" free list structure, because you need to validate? Is it a good idea to chop up all the space available, so you can validate different parts independently, or does that mean you end up with the multi-record transaction problem? These are the kinds of things we consider in database design on new hardware ( obligatory: we are hiring ).
scurvy 9y ago
What's a "high-NVMe chassis" and what large cloud provider uses Dell gear these days? They long ago killed the C-series line as everyone switched to Quanta (FB et al) or Supermicro (smaller players). The only people buying Dell are enterprises, and they don't buy in C-series volume nor are they price conscious.
- deleted 9y ago
  [deleted]
- bbulkow 9y ago
  The truely humongous buyers, like Amazon / Google / FB, certainly don't buy dell anymore, also don't buy software from people like Aerospike. Medium-size companies ( say, box, or AppLovin, or PayPal ) buy software, have some cost consciousness, and buy Dell. I expect Dell will come out with some chassis closer to what Supermicro has been offering to the market, I also see some defection just like you do. However, that's all speculation.

bch 9y ago

> All of that means you have to code to it differently. > That requires new coding, we're on it. Can you give a rough example to guide my thinking? I understand OSes are considering ways that this has both traditional disk and RAM properties, and are sorting out storage subsystems of drivers, but I assume you're talking about something closer to user-level structures and algorithms?

wtallis 9y ago
Databases have traditionally gone to great lengths to bypass whatever heuristics the operating system is using for I/O scheduling and caching so that they can implement their own optimizations while maintaining their data consistency guarantees. Any changes to OS behavior you can imagine are also potentially applicable to databases that take this approach.
bbulkow 9y ago
Yes, I think there are clear user-level approaches you can ( and must ) take. There are some interesting talks out there about NAND, how it works, and how to optimize - I saw something here on HN a few days ago about writing your own time-series database, which got a variety of the facts wrong but was an example of how to choose data structures that are NAND-reasonable. You can look up some of my YouTube talks and slideshare, for example - I've been talking this for a while. At a high level, NAND has more IOPs than god, because they don't seek. An old enterprise spindle littlerally does 200 to 250 seeks per second. And Flash can read from 500,000 different random locations per second. That's so far apart that different user level approaches are called for. In terms of XPoint, let me give you one detail. What does a "commit" look like in XPoint? What do the different kinds of memory barriers look like? What's the best way to validate this kind of persistence on restart, which you don't have to do with DRAM? Does that change your "malloc" free list structure, because you need to validate? Is it a good idea to chop up all the space available, so you can validate different parts independently, or does that mean you end up with the multi-record transaction problem? These are the kinds of things we consider in database design on new hardware ( obligatory: we are hiring ).

scurvy 9y ago

What's a "high-NVMe chassis" and what large cloud provider uses Dell gear these days? They long ago killed the C-series line as everyone switched to Quanta (FB et al) or Supermicro (smaller players). The only people buying Dell are enterprises, and they don't buy in C-series volume nor are they price conscious.

deleted 9y ago
[deleted]
bbulkow 9y ago
The truely humongous buyers, like Amazon / Google / FB, certainly don't buy dell anymore, also don't buy software from people like Aerospike. Medium-size companies ( say, box, or AppLovin, or PayPal ) buy software, have some cost consciousness, and buy Dell. I expect Dell will come out with some chassis closer to what Supermicro has been offering to the market, I also see some defection just like you do. However, that's all speculation.