4 ms·
There are several reasons 1. GPU RAM storage isn't as fine grained as CPU virtual memory. While GPU's have virtual memory they don't have the same degree of Co
by valarauca1 10y ago
There are several reasons
1. GPU RAM storage isn't as fine grained as CPU virtual memory. While GPU's have virtual memory they don't have the same degree of Copy On Write hardware utilities. This make rolling back snapshots and transactions within memory very difficult.
2. GPU's don't have dedicated non-volatile storage (well some do, but it is used more as a cache, and is treated as volatile). So for loading data you:
SSD -> RAM -> CPU *copy* CPU -> RAM-> GPU
This isn't really a question of technological maturity it is more a question of PCIe doesn't allow an NVMe SSD to talk directly to a PCIe GPU. Nor do modern OS's have any model how to do this, nor do GPU's support file systems.
3. SQL based data querying is parallel friendly. At it's core SQL is 99% Filter/Map operations. The original goal of SQL was to be bottle necked by HDD access times so the vast majority of the query work is done in O(n). With a
SSD -> RAM -> CPU *copy* CPU -> RAM-> GPU
You are already paying that O(n) load+process price at copy time (minus branches).
So the savings are only present if the data can persist on the GPU. Ultimately having multiple CPU threads to do the Map/Filter do the same job
There isn't just one problem:
1. GPU's don't support features to make holding data in RAM useful
2. PCIe doesn't support features to make loading data into the GPU fast.
3. The very design of SQL makes copying data into the GPU moot. Spreading a Map/Filter over 10-20 threads isn't rocket science.
- tmostak 10y agoYour first point is an implementation detail. Why couldn't CPU RAM be used as well in a GPU database system? Re CPU vs GPU performance you are neglecting the fact that GPU RAM can be an order-of-magnitude faster than CPU RAM, not to mention the fact that complex queries can often become compute bound (geospatial queries are a prominent example).
- valarauca1 10y agoWhy couldn't CPU RAM be used as well in a GPU database system? There is no market demand for this currently. GPU's don't need advanced MMU's because nobody wants them. The same is true for PCIe data loading. CPU vs GPU performance you are neglecting the fact that GPU RAM can be an order-of-magnitude faster than CPU RAM, No I'm not. If your data set can remain in GPU ram long term yes there is a big performance speed up. The problem here (as I outlined before) 1. GPU MMU's are less advanced, so they don't handle transactions well (see above comments). 2. As you can't perform transactions efficiently, you need read only data. Which is possible but many databases aren't read only, or databases NOT being written too are rare. 3. Ensuring your full dataset can fit in GPU memory (typically <20GB) is a damning limitation. Streaming data into the GPU is rather sloppy and limits compute though-put.
- nine_k 10y agoThe CPU / RAM is a bottleneck between the persistent storage and the fast GPU. There's a well-known way to increase network throughput and reduce latency by running a whole dedicated IP stack + hardware driver in the user space of the process that needs it. This removes the bottleneck of the OS kernel. I wonder if there's a way to remove the bottleneck in the GPU case by something similar, by dedicating a piece of hardware to the SSD interconnect without having a CPU as an intermediary. AFAICT you still cannot DMA from a disk directly to GPU RAM, even though you have enough PCIe lanes coming to the GPU. Is this true? If so, can it change in a way that is still compatible with traditional, "normal" operation of the PC architecture?