3 ms·

What the what? 10 times faster for one join? https://wiki.postgresql.org/images/5/50/PGStrom_Fig_MicroBenchGpuJoin.png https://wiki.postgresql.org/images/5/50

by ck2 11y ago

What the what? 10 times faster for one join? https://wiki.postgresql.org/images/5/50/PGStrom_Fig_MicroBenchGpuJoin.png https://wiki.postgresql.org/images/5/50/PGStrom_Fig_MicroBen... Or am I reading that incorrectly? How is that possible?

lazyjones 11y ago
> How is that possible? I'll guess: It's all in-memory, so not I/O-bound. Rows are processed in parallel with PGStrom (in 15MB chunks), while Postgres does it sequentially in one thread. Apparently moving the data around is fast enough not to matter.
mslot 11y ago
Joining a large number of tables is a sweet spot for PGStrom where in which there are a lot of parallelizable computations. For a lot of queries it can actually be slower.
- calpaterson 11y ago
  Hopefully the query planner would choose whether or not to use PGStrom for a specific query. Many interesting queries in normalised databases involve a lot of joins. In a BCNF database most interesting queries use 3 or more joins. Whether or not I believe the microbenchmarks is another thing :)
dsugarman 11y ago
Think of GPUs as excelling at MapReduce, either they do the same operation on many data points or they reduce the data by combining them in an aggregate. The query listed is taking an avg on a variable # of joins. Each join is doing a sequential search which can be a lot faster if you can process this in parallel and then when you find the right data you want to aggregate, you are then doing the aggregate. Both are incredibly faster in parallel. The gpu was designed so you can, lets say, move all vertices in a character's body 1 inch forward, you would need to process this one translation for potentially thousands of vertices. It was also designed to do bitmap operations very quickly where you need to get the same image, but smaller so the GPU would combine nearby pixels to provide a good heuristic of what that same image would look like further away.

by ck2 11y ago

lazyjones 11y ago

> How is that possible? I'll guess: It's all in-memory, so not I/O-bound. Rows are processed in parallel with PGStrom (in 15MB chunks), while Postgres does it sequentially in one thread. Apparently moving the data around is fast enough not to matter.

mslot 11y ago

Joining a large number of tables is a sweet spot for PGStrom where in which there are a lot of parallelizable computations. For a lot of queries it can actually be slower.

calpaterson 11y ago
Hopefully the query planner would choose whether or not to use PGStrom for a specific query. Many interesting queries in normalised databases involve a lot of joins. In a BCNF database most interesting queries use 3 or more joins. Whether or not I believe the microbenchmarks is another thing :)

dsugarman 11y ago

Think of GPUs as excelling at MapReduce, either they do the same operation on many data points or they reduce the data by combining them in an aggregate. The query listed is taking an avg on a variable # of joins. Each join is doing a sequential search which can be a lot faster if you can process this in parallel and then when you find the right data you want to aggregate, you are then doing the aggregate. Both are incredibly faster in parallel. The gpu was designed so you can, lets say, move all vertices in a character's body 1 inch forward, you would need to process this one translation for potentially thousands of vertices. It was also designed to do bitmap operations very quickly where you need to get the same image, but smaller so the GPU would combine nearby pixels to provide a good heuristic of what that same image would look like further away.