5 ms·
Several companies have implemented databases on GPUs but there is a good technical reason that the approach has never really caught on, and some of these compan
by jandrewrogers 10y ago
Several companies have implemented databases on GPUs but there is a good technical reason that the approach has never really caught on, and some of these companies even migrated to selling the same platform on CPUs only.
The weakness of GPU databases is that while they have fantastic internal bandwidth, their network to the rest of the hardware in a server system is over PCIe, which generally isn't going to be as good as what a CPU has and databases tend to be bandwidth bound. This is a real bottleneck and trying to work around it makes the entire software stack clunky.
I once asked the designer of a good GPU database what the "no bullshit" performance numbers were relative to CPU. He told me GPU was about 4x the throughput of CPU, which is very good, but after you added in the extra hardware costs, power costs, engineering complexity etc, the overall economics were about the same to a first approximation. And so he advised me to not waste my time considering GPU architectures for databases outside of some exotic, narrow use cases where the economics still made sense. Which was completely sensible.
For data intensive processing, like databases, you don't want your engine to live on a coprocessor. No matter how attractive the coprocessor, the connectivity to the rest of the system extracts a big enough price that it is rarely worth it.
- kusmi 10y agoIs the data stored in GPU memory?
- VodkaHaze 10y agoLimiting your queries to ~12gb, the max ram on one GPU, (beyond which PCIe I/O becomes a bottleneck) will be a problem for business use, I'd think
- j1vms 10y agoFor some reason I am surprised that anyone would want to ship the data whole and as is, to the GPU. Wouldn't it make more sense to use a representative, transformed "GPU-ready" data set, both much smaller in size & designed specifically for the queries that are to be optimized?
- Johnny555 10y agoOnce you read and prune out the dataset to only include the relevant data, then what's left for the GPU to do?
- j1vms 10y agoThe transformation to GPU-ready would not be as trivial and effectively redundant as pruning the data of course. It would be produce a secondary data structure, like an index on a column, though in this case of course destined to be processed within the math-oriented, high-branch-cost setting of a GPU.
- lmeyerov 10y agoThis is basically the bread and butter of columnar compute systems, not just GPU ones. GPUs ones just get to throw more compute at them, and thus do even better for these kinds of space/time trade-offs. Interestingly, most big data systems are increasingly columnar.
- felipe_aramburu 10y agoWe are not shipping all of the data as a whole to the gpu. We are going to be releasing some whitepapers that explain this in more detail but lets get a few things clear. Data is sent to the gpu compressed since it is compressed when it is stored. We can decompress VERY quickly on GPU's (30-50GB/s is easily achievable on a K80) and because each of our columns are compressed using one of our cascading compression algorithms (which everyone offers the best in terms of compression and throughput). We are a column store and only send over the columns that are being used in processing. So for example select id, name, age, avg(income) from people group by gender In this case only the income and gender columns would actually be sent to the gpu and they would do so in a compressed fashion to increase the "effective" bandwidth of data over PCIE. Even more interesting is that id, name, and age, would be pulled from our horizontal store instead of our compressed columnar store in order to minimize the number of iops necessary to fill the result set.
- deleted 10y ago[deleted]
- felipe_aramburu 10y agoI am the CTO of blazingDB. The data is not stored in GPU or even RAM. We operate from the disk though we we will cache information in RAM when we have plenty of it available.
- tw04 10y agoIt would seem that rationalization would fall apart quickly with the new Power CPUs that have NVLink built right into the CPU. Getting data back and forth shouldn't be a problem anymore. Outside of CPU >> GPU, I'm not sure what other data movement you could be talking about. A SAS HBA or Ethernet NIC or Infiniband HBA are almost always going to be operating over the same PCIe bus the GPU uses. In the rare instances they're built onto the CPU, the "network link" is likely going to still be significantly slower than the fastest PCIe slot.
- marcell 10y agoWhat is NVLink, and what does it mean in terms of data transfer?
- fulafel 10y agoIt's a faster link between between future IBM POWER processors and NVidia GPUs, but won't make waves in database market since those systems are niche HPC/supercomputing hardware.
- jzelinskie 10y agoI've also seen NVLink on some of the pre-Pascal roadmaps for nVidia's gaming-oriented graphics cards. Since the current generation gaming consoles has HSA, I'm hoping that it gains in popularity and because less niche. Problem is that it'd be a pretty vital component to be nVidia proprietary.
- snuxoll 10y agoCAPI is "open" solution to do the same thing, though NVLink supposedly still offers more bandwidth. Unfortunately CAPI is only available on POWER8 hardware at the moment and I'm not sure if IBM is open to license it beyond OpenPOWER - still, there's a decent number of CAPI-capable cards already available and more coming to the market in the future. GPU's may not be great for every unit of work a RDBMS has to perform, but given their ability to rapidly compute hashes it could help a lot with joins (as evidenced by PGStrom).
- velox_io 10y agoYou pretty much hit the nail on the head, a big limitation is the feeding the GPUs in the first place. Furthermore, all that memory bandwidth is calculated against all the cores. So you have to be VERY careful in usage patterns (it doesn't work like a giant CPU). Not to mention how much it costs involved per GB or TB! I have some experience in this area, and where GPU's & database really shine is building (and especially) re-building indexes. Running a database on GPUs isn't going to replace all DBs overnight. But, the hardware does have it's uses; just like improved SIMD on CPU's and NVMe storage, plus developments in networking - Basically look where Intel is going since we're coming to the limits of silicon transistors..
- _asummers 10y agoI can definitely see value in offloading a bunch of number crunching on sufficiently large data sets to a graphics card (particularly parts that are trivially parallelizable). But to me, that seems like an optimization a database would make in its query planner, not one that you'd necessarily want to build an entire standalone product around.
- dharma1 10y agoWhen did you ask? GPUs have been getting a lot faster year on year. Though, I would like to see Xeon Knights Landing compared to GPUs for db uses.
- dspillett 10y agoThe data bandwidth between main memory and that which the GPU works with isn't speeding up by the same factors though. This is fine when working with a dataset that fits into the GPU's memory pool and your workload involves relatively few (or zero) changes because you can transfer it once and repeatedly ask the GPUs to analyse it in what-ever ways. As soon as the common dataset doesn't fit neatly into the GPU's RAM (leaving enough spare for scratch space) you end up thrashing the channel and it becomes the main bottleneck.
- dharma1 10y agoThat's true. There are some workarounds though, like distributing the workload between multiple GPUs - actively under development in machine learning for instance
- gaius 10y agoyou don't want your engine to live on a coprocessor Well yes and no. Both IBM and Oracle have gotten impressive performance using database-specific co-processors, but these are not GPGPUs, they are dedicated hardware that sits on the storage path. Baidu are reinventing that wheel with FPGAs too.
- mcphage 10y ago> The weakness of GPU databases is that while they have fantastic internal bandwidth, their network to the rest of the hardware in a server system is over PCIe, which generally isn't going to be as good as what a CPU has and databases tend to be bandwidth bound. This is a real bottleneck and trying to work around it makes the entire software stack clunky. How relevant is that when you're looking at multi-TB data sets that don't fit into computer RAM? Sure, the RAM <---> CPU bandwidth may be very wide, but the SSD connects to the computer over the same PCIe bus. And also: when did you have this conversation? GPU performance has changed very much year by year, so what wouldn't have been worth it 2 years ago might be a huge gain now.
- scott_s 10y agoThe difficulty with how most GPUs are connected to the rest of the system is that the data has to go RAM -> CPU -> GPU. If it could go directly RAM -> GPU, then the calculations would be better, but still not great as PCIe is still lower bandwidth and higher latency than RAM -> CPU. It's not about GPU performance, it's about the latency and bandwidth of getting that dat to the GPU. If once you ship data to the GPU, you reuse it many times for many calculations, that cost is amortized and it doesn't matter as much. But if you ship data to the GPU and use it once, then that cost will probably not be amortized. I think of databases tending to fit in the latter category.
- scott_s 10y agoFully agreed. I wrote a paper six years ago that came to the same conclusion: http://www.scott-a-s.com/files/debs2010.pdf http://www.scott-a-s.com/files/debs2010.pdf