5 ms·
It's not that simple. The FTL addresses two fundamental problems with flash - slow random write performance and wear leveling. It works beautifully on consumer
by coffeemug 17y ago
It's not that simple. The FTL addresses two fundamental problems with flash - slow random write performance and wear leveling. It works beautifully on consumer hardware, but performs rather poorly in an enterprise environment. For some workloads you can squeeze a lot more than 5-10% improvement out of the drives. But that's not the interesting part (most of today's workloads are read-intensive anyway).
The interesting part comes from being able to execute existing operations in fundamentally different ways. When you're not restricted to sequential reads you can implement a huge chunk of database internals differently - different concurrency models, different implementations of certain joins, etc. This is where the bulk of the performance improvement comes from.
Once you do this, you realize that a traditional bottleneck (I/O) is completely blown away. Suddenly, all kinds of things nobody ever bothered to optimize become performance bottlenecks (for example, the query plan is traditionally interpreted, not compiled to native code - not good enough once you get an I/O subsystem that's 250 times faster than what you used to have).
Once you get rid of the software bottlenecks, you realize that you're limited by the bus. Since SSDs are highly parallelized devices, you have to change the way you interleave operations for optimal performance. That alone is another order of magnitude.
In three years or so, the flash manufacturers will put chips on the SSDs that will allow you to send down programs to them (not unlike the shaders in video cards). That will likely be a bigger revolution than SSDs themselves. Where we are now is just the beginning.
You're right that we haven't shown any evidence that our approach is anything but a niche. We will. This stuff just takes a long time to build.
- neilc 17y agothe query plan is traditionally interpreted, not compiled to native code Actually, System R (IBM's pioneering relation database from the mid 1970s) compiled query plans down to machine code.
- pedrocr 17y agoIn three years or so, the flash manufacturers will put chips on the SSDs that will allow you to send down programs to them (not unlike the shaders in video cards). That will likely be a bigger revolution than SSDs themselves. Where we are now is just the beginning. This sounds crazy. SSDs have all this firmware and processing power to make them behave like hard drives. If we want to drive the underlying storage directly we might as well rip all that out and let the OS drive the flash directly from the main CPU. Why would we want to be sending programs into the black box? What can the firmware do that the main CPU couldn't if we stopped attaching flash as SATA and instead just created a direct connection that can DMA back and forth to main memory? Some Linux developers have made that point, that they would rather be implementing their own filesystems directly on the flash than dealing with a firmware that is trying to make it appear that the SSD is a hard drive but that then needs TRIM to work around the fact that that is not true. These firmwares are nice for Intel by letting them turn a commodity (flash) into a differentiated (and higher margin) product based on the code in the firmware. Considering Linux already has filesystems that can operate directly over flash and that better ones can be written I'd say we would be better off just getting direct access to the flash.
- DougWebb 17y agoI agree with your point in general, but if coffeebus' assertion that SSD access will become limited by the bus, it still makes sense to allow programs to be sent to run directly on the SSD. Remember, video cards originally were just big ram buffers that were accessed directly by the CPU, but that wasn't fast enough so now we have shaders on the cards. Having something similar in an SSD will allow a lot of data searching and crunching to happen on the SSD, and it'll run in parallel with whatever else the CPU might be busy doing.
- igrigorik 17y agoI see your point, but I'm not convinced that the amount of data (raw volume size) and the read speeds of flash lead to the same behavior as shaders + GPU. GPU's are processing an order (or several) of magnitude more data, which makes shaders a useful feature: data and computation locality. Flash is 10x slower than RAM (in an ideal case), and even then, we haven't had the need to introduce 'shader like' features into RAM - yes, because historically we haven't stored all that much in it, but even with modern 64Gb configurations, same thing...
- DougWebb 17y agoI don't think the comparison is Flash vs RAM, I think it's "directly-accessed Flash" vs "Flash on the other side of a PCI-E bus". We don't need anything like a shader for RAM because the bus RAM is attached to is fast. The question at hand is whether the Flash on an SDD is best accessed as a SATA device, DMA Flash over PCI-E (or some new slot technology), or through shader-like cpus on the same card as the Flash.