3 ms·
Is processing speed the bottleneck for databases?
by ape4 10y ago
Is processing speed the bottleneck for databases?
- remus 10y agoIt depends what you're doing. I haven't looked in a while, but last time I did the idea behind MapD was to produce interactive visualisations over lots of data. That can certainly get pretty computationally expensive.
- random3 10y agoThat's the big question I have too. I think typically it's I/O in one form or another. With GPUs sending data to the GPU seems to be another cost in the path. However if you consider the GPU part as a way to push down processing power close to data, then you can probably solve more complex problems like graph processing efficiently within the DB..
- jandrewrogers 10y agoBandwidth for moving data is typically the limiting factor if you are doing everything correctly, very rarely computation. Bottlenecks for workloads are commonly memory or network bandwidth these days.
- manigandham 10y agoIt can be. It's a complex balance between processing speed and IO (from disk, ram and network). GPUs are much more parallel and a great fit for most queries which can be broken up, so combine that with very fast GPU ram and local storage and you can eliminate network/distributed fetches to make scans even faster.
- detaro 10y agoGPUs can also have better memory bandwidth, if the relevant parts of the dataset fit in GPU memory and the access patterns work out. Which could happen for clever index structures and things like that. Or complex processing over limited results of (sub-)queries.
- tmostak 10y agoGPUs have phenomenal memory bandwidth (multiple terabytes per second in a multi-GPU server) in addition to the computational parallelism they are known for. MapD caches the most recently touched data in the VRAM across multiple GPUs, meaning it can execute scan queries over billions of records in milliseconds.