5 ms·

This page [1] lists query times on a 1.1 billion row dataset, performed using a variety of systems, from GPU DBs all the way to SQLite. [1] https://tech.marks

by scribu 8y ago

This page [1] lists query times on a 1.1 billion row dataset, performed using a variety of systems, from GPU DBs all the way to SQLite. [1] https://tech.marksblogg.com/benchmarks.html https://tech.marksblogg.com/benchmarks.html Edit: Unfortunately, each run is on different hardware, but it at least gives you an idea of what's possible.

walterbell 8y ago
Anyone used this GPU database that ranks at the top of that list, above the open-source MapD? https://www.brytlyt.com https://www.brytlyt.com
- bufferoverflow 8y ago
  Note that it's at the top of the list probably because it's running on a cluster. It would be awesome to see such a comparison on some standard hardware, like a large AWS GPU instance (eg1.2xlarge). Also note that the dataset is 600GB, so it won't fit a sinlge GPU, not even close.
- datumdadum 8y ago
  Haven't, but it's worth noting that hardware is probably attributable to them edging out Mapd since they're on a 5-node minsky cluster featuring nvlink, hence as arnon said, are benefiting from 9.5x faster transfer from disk than than PCIe 3.0. That blog has not yet tested Mapd's IBM Power version-- would be interesting to see how it compared on that cluster. CMU has an interesting lecture if you want to learn more about Brytlyt https://www.youtube.com/watch?v=oL0IIMQjFrs https://www.youtube.com/watch?v=oL0IIMQjFrs Couple interesting things to note: - They attain some of their speed by requiring that data be pre-sorted https://youtu.be/oL0IIMQjFrs?t=1092 https://youtu.be/oL0IIMQjFrs?t=1092 https://youtu.be/oL0IIMQjFrs?t=1127 https://youtu.be/oL0IIMQjFrs?t=1127 - They've built their database on Postgres for query planning, but for any query which does not match what they've accelerated on GPU, they do not have the ability to failover to utilizing postgres on the CPU. https://youtu.be/oL0IIMQjFrs?t=3260 https://youtu.be/oL0IIMQjFrs?t=3260 - Data is brought into GPU memory at table CREATE time, so the cost of transferring data from disk->host RAM->GPU RAM is not reflected. Probably wouldn't work if you want to shuffle data in/out of GPU RAM across changing query workloads. https://youtu.be/oL0IIMQjFrs?t=1310 https://youtu.be/oL0IIMQjFrs?t=1310 - The blog had to use data type DATE instead of DATETIME for Brytlyt since it doesn't support the latter. DATETIME was used for the other DBs, which is a heavier computation. https://tech.marksblogg.com/billion-nyc-taxi-rides-p2-16xlarge-brytlytdb-2.html https://tech.marksblogg.com/billion-nyc-taxi-rides-p2-16xlar... So all-in-all it seems like a more carefully constructed, hardware balanced comparison would be needed to see which the quickest would be.
arnon 8y ago
Also worth noting this is a dataset that fits comfortably in-memory
- Rapzid 8y ago
  And the Postgres run was on 16GB of RAM and a rather slow SSD in a single drive configuration. Would have been interesting to see the results of either in memory or on a faster storage system.
sacheendra 8y ago
The cost of GPUs doesn't make sense for the compute they offer. According the benchmark, the fastest 8 GPU node takes about 0.5 seconds. The cost of that node on AWS is about 24$/hour. The 21 node spark cluster takes 6 seconds. But, it only costs 4$/hour. An additional benefit with Spark is that it can be used for a lot more variety of operations than a GPU. This cost disadvantage restricts GPU processing to niche use cases.
- TomVDB 8y ago
  > According the benchmark, the fastest 8 GPU node takes about 0.5 seconds. The cost of that node on AWS is about 24$/hour. The 21 node spark cluster takes 6 seconds. But, it only costs 4$/hour. Using your numbers, the GPU solution has half the cost for similar performance? How does that not make sense? > This cost disadvantage restricts GPU processing to niche use cases. All GPU compute applications are niche use cases.
  - johnvanommen 8y ago
    Look man, you don't get it. The GPU case is half the cost, but it's also twelve times faster. Oh, wait...
- latchkey 8y ago
  > The cost of GPUs doesn't make sense for the compute they offer. This assumes AWS pricing. You build a farm of GPUs and buy in bulk, you get much better cost basis. GPU farms are becoming more and more of a thing now and definitely less 'niche'.

by scribu 8y ago

walterbell 8y ago

Anyone used this GPU database that ranks at the top of that list, above the open-source MapD? https://www.brytlyt.com https://www.brytlyt.com

bufferoverflow 8y ago
Note that it's at the top of the list probably because it's running on a cluster. It would be awesome to see such a comparison on some standard hardware, like a large AWS GPU instance (eg1.2xlarge). Also note that the dataset is 600GB, so it won't fit a sinlge GPU, not even close.
datumdadum 8y ago
Haven't, but it's worth noting that hardware is probably attributable to them edging out Mapd since they're on a 5-node minsky cluster featuring nvlink, hence as arnon said, are benefiting from 9.5x faster transfer from disk than than PCIe 3.0. That blog has not yet tested Mapd's IBM Power version-- would be interesting to see how it compared on that cluster. CMU has an interesting lecture if you want to learn more about Brytlyt https://www.youtube.com/watch?v=oL0IIMQjFrs https://www.youtube.com/watch?v=oL0IIMQjFrs Couple interesting things to note: - They attain some of their speed by requiring that data be pre-sorted https://youtu.be/oL0IIMQjFrs?t=1092 https://youtu.be/oL0IIMQjFrs?t=1092 https://youtu.be/oL0IIMQjFrs?t=1127 https://youtu.be/oL0IIMQjFrs?t=1127 - They've built their database on Postgres for query planning, but for any query which does not match what they've accelerated on GPU, they do not have the ability to failover to utilizing postgres on the CPU. https://youtu.be/oL0IIMQjFrs?t=3260 https://youtu.be/oL0IIMQjFrs?t=3260 - Data is brought into GPU memory at table CREATE time, so the cost of transferring data from disk->host RAM->GPU RAM is not reflected. Probably wouldn't work if you want to shuffle data in/out of GPU RAM across changing query workloads. https://youtu.be/oL0IIMQjFrs?t=1310 https://youtu.be/oL0IIMQjFrs?t=1310 - The blog had to use data type DATE instead of DATETIME for Brytlyt since it doesn't support the latter. DATETIME was used for the other DBs, which is a heavier computation. https://tech.marksblogg.com/billion-nyc-taxi-rides-p2-16xlarge-brytlytdb-2.html https://tech.marksblogg.com/billion-nyc-taxi-rides-p2-16xlar... So all-in-all it seems like a more carefully constructed, hardware balanced comparison would be needed to see which the quickest would be.

arnon 8y ago

Also worth noting this is a dataset that fits comfortably in-memory

Rapzid 8y ago
And the Postgres run was on 16GB of RAM and a rather slow SSD in a single drive configuration. Would have been interesting to see the results of either in memory or on a faster storage system.

sacheendra 8y ago

The cost of GPUs doesn't make sense for the compute they offer. According the benchmark, the fastest 8 GPU node takes about 0.5 seconds. The cost of that node on AWS is about 24$/hour. The 21 node spark cluster takes 6 seconds. But, it only costs 4$/hour. An additional benefit with Spark is that it can be used for a lot more variety of operations than a GPU. This cost disadvantage restricts GPU processing to niche use cases.

TomVDB 8y ago
> According the benchmark, the fastest 8 GPU node takes about 0.5 seconds. The cost of that node on AWS is about 24$/hour. The 21 node spark cluster takes 6 seconds. But, it only costs 4$/hour. Using your numbers, the GPU solution has half the cost for similar performance? How does that not make sense? > This cost disadvantage restricts GPU processing to niche use cases. All GPU compute applications are niche use cases.
- johnvanommen 8y ago
  Look man, you don't get it. The GPU case is half the cost, but it's also twelve times faster. Oh, wait...
latchkey 8y ago
> The cost of GPUs doesn't make sense for the compute they offer. This assumes AWS pricing. You build a farm of GPUs and buy in bulk, you get much better cost basis. GPU farms are becoming more and more of a thing now and definitely less 'niche'.