6 ms·

Blazegraph 2.0 – GPU-accelerated distributed graph database

103 points by espeed 11y ago

zzleeper 11y ago
ERR_CONNECTION_TIMED_OUT
- hendler 11y ago
  Meanwhile, check out: https://github.com/blazegraph/database Also down: https://blog.blazegraph.com/?p=977 https://wiki.blazegraph.com/
- beebs 11y ago
  Should be all resolved. Tuning EC-2 up for the /. effect...
espeed 11y ago
MapGraph -- the GPU-accelerated graph engine -- has been rolled into Blazegraph 2.0, and it looks like this means OLTP and OLAP can be combined into a blazing-fast, single OLXP system. From: https://www.blazegraph.com/product/gpu-accelerated/ https://www.blazegraph.com/product/gpu-accelerated/ The original work was funded by DARPA and presented at the 2014 SIGMOD conference in a paper entitled, MapGraph: A High Level API for Graphs [1]. This work is available in open source. Later work, in collaboration with the University of Utah SCI Institute [2] and funded by DARPA, applied multi-core techniques running on over 750 M cores on the Titan Supercomputer to extend this to Multi-GPU traversal with Breadth First Search (BFS). On a cluster of 64 NVIDIA K40 GPUs, it demonstrated a throughput of 32 Billion Traversed Edges Per Second (32 GTEPS), traversing a scale-free graph of 4.3 billion directed edges in 0.15 seconds, which was featured in a presentation IEEE Bigdata Conference. [1] https://www.blazegraph.com/whitepapers/MapGraph-SIGMOD-2014.pdf https://www.blazegraph.com/whitepapers/MapGraph-SIGMOD-2014.... [2] http://www.sci.utah.edu/publications/Fu2014a/UUSCI-2014-002.pdf http://www.sci.utah.edu/publications/Fu2014a/UUSCI-2014-002.... VIDEO: Blazegraph GPU and DASL at Super Computing 2015 (https://vimeo.com/148519808 https://vimeo.com/148519808) It's GPLv2 and now has support for SPARQL and TinkePop3/Gremlin. See https://groups.google.com/d/topic/gremlin-users/8fS_ak-tWNs/discussion https://groups.google.com/d/topic/gremlin-users/8fS_ak-tWNs/...
hmottestad 11y ago
We've used Blazegraph at work for a project, and we have used a number of other graph databases. Blazegraph is a very niche product and requires a lot of time for setting it up and adjusting it for your workload. If Blazegraph peaks your interest then you should also look into Yarc platform by Cray. Should you want to look more into graphs, but don't want to spend endless nights just trying to load your data then I would recommend Stardog, which has just been a pleasure to work with.
- kendallgclark 11y ago
  http://stardog.com/ http://stardog.com/
  - vdaniuk 11y ago
    Stardog is great and would be even better with a friendly licensing policy towards indie developers. Their community version is too limited and enterprise version price is per server, contact sales. Current developer version is for testing only. I would happily pay for a developer license or something like that.
    - kendallgclark 11y ago
      We sell Developer licenses to developers, indie or not, all the time. Send email to sales@complexible.com! :)
      - vdaniuk 11y ago
        Great news, I will. I guess that option wasn't obvious for me when I was looking over http://stardog.com/#plans http://stardog.com/#plans
- pudo 11y ago
  The promises they make are really tempting. However, when I experimented with Blazegraph last year, it ended up stuck more often than I could count, whether during bulk import or in answering some ad-hoc SPARQL queries. I can imagine that with extensive tuning, BlazeGraph provides a good database. Just don't expect it to have the polish and convenience of a modern RDBMS or a shiny NoSQL store :)
danbee 11y ago
"Your connection is not secure The owner of www.blazegraph.com has configured their website improperly. To protect your information from being stolen, Firefox has not connected to this website."
- beebs 11y ago
  We've seen cases where the newer Thawte root certs were not included in the trust chain. That might be the issue.
BenoitP 11y ago
> provides a Scala-based language to write graph and big data analytics and is complementary to the Spark and Hadoop ecosystems It'd be awesome to have Blazegraph as a backend for Spark's Pregel queries. With Tensorflow bindings in place, and the BIDMach/BIDMat libraries, it is very nice seeing Spark getting some serious GPU attention.
- beebs 11y ago
  We definitely see Spark + Scala + Blazegraph DASL to be a sweet spot for combining the ease of Spark with the GPU performance. Have a submission in to the Hadoop Summit in June on it: https://hadoopsummit.uservoice.com/forums/344955-data-science-analytics-and-spark/suggestions/11664123-the-power-of-spark-the-speed-of-gpus-for-graph-an https://hadoopsummit.uservoice.com/forums/344955-data-scienc...
  - rxin 11y ago
    You guys should submit a talk to Spark Summit. Look forward to it.
beebs 11y ago
Got any questions on Blazegraph GPU? Just let us know...
- michaelt 11y ago
  What's the performance like on data sets that are too big for any single GPU's memory? Can the GPU be used to accelerate shortest-path queries (e.g. dijkstra's algorithm) and if so, where can I read more about how that's achieved?
  - beebs 11y ago
    The graph does need to fit into GPU ram. We use graph partitioning for multi-node, Multi-GPU configurations. Dijkstra's algorithm which, as mentioned by Davidson et al. [1], is a "sequential algorithm [that] is poorly suited for parallel architectures like GPUs that require large numbers of parallel threads for efficient execution." Instead, we have variants of the algebraic formulation of the Bellman-Ford algorithm as given in Kepner and Gilbert's book [2]. [1] Andrew A. Davidson, Sean Baxter, Michael Garland, and John D. Owens: "Work-Efficient Parallel GPU Methods for Single-Source Shortest Paths." In Proceedings of the IEEE 28th International Parallel and Distributed Processing Symposium (IPDPS), 2014. http://dx.doi.org/10.1109/IPDPS.2014.45 http://dx.doi.org/10.1109/IPDPS.2014.45 [2] Kepner and Gilbert: "Graph Algorithms in the Language of Linear Algebra."

103 points by espeed 11y ago

zzleeper 11y ago

ERR_CONNECTION_TIMED_OUT

hendler 11y ago
Meanwhile, check out: https://github.com/blazegraph/database Also down: https://blog.blazegraph.com/?p=977 https://wiki.blazegraph.com/
beebs 11y ago
Should be all resolved. Tuning EC-2 up for the /. effect...

espeed 11y ago

MapGraph -- the GPU-accelerated graph engine -- has been rolled into Blazegraph 2.0, and it looks like this means OLTP and OLAP can be combined into a blazing-fast, single OLXP system. From: https://www.blazegraph.com/product/gpu-accelerated/ https://www.blazegraph.com/product/gpu-accelerated/ The original work was funded by DARPA and presented at the 2014 SIGMOD conference in a paper entitled, MapGraph: A High Level API for Graphs [1]. This work is available in open source. Later work, in collaboration with the University of Utah SCI Institute [2] and funded by DARPA, applied multi-core techniques running on over 750 M cores on the Titan Supercomputer to extend this to Multi-GPU traversal with Breadth First Search (BFS). On a cluster of 64 NVIDIA K40 GPUs, it demonstrated a throughput of 32 Billion Traversed Edges Per Second (32 GTEPS), traversing a scale-free graph of 4.3 billion directed edges in 0.15 seconds, which was featured in a presentation IEEE Bigdata Conference. [1] https://www.blazegraph.com/whitepapers/MapGraph-SIGMOD-2014.pdf https://www.blazegraph.com/whitepapers/MapGraph-SIGMOD-2014.... [2] http://www.sci.utah.edu/publications/Fu2014a/UUSCI-2014-002.pdf http://www.sci.utah.edu/publications/Fu2014a/UUSCI-2014-002.... VIDEO: Blazegraph GPU and DASL at Super Computing 2015 (https://vimeo.com/148519808 https://vimeo.com/148519808) It's GPLv2 and now has support for SPARQL and TinkePop3/Gremlin. See https://groups.google.com/d/topic/gremlin-users/8fS_ak-tWNs/discussion https://groups.google.com/d/topic/gremlin-users/8fS_ak-tWNs/...

hmottestad 11y ago

We've used Blazegraph at work for a project, and we have used a number of other graph databases. Blazegraph is a very niche product and requires a lot of time for setting it up and adjusting it for your workload. If Blazegraph peaks your interest then you should also look into Yarc platform by Cray. Should you want to look more into graphs, but don't want to spend endless nights just trying to load your data then I would recommend Stardog, which has just been a pleasure to work with.

kendallgclark 11y ago
http://stardog.com/ http://stardog.com/
- vdaniuk 11y ago
  Stardog is great and would be even better with a friendly licensing policy towards indie developers. Their community version is too limited and enterprise version price is per server, contact sales. Current developer version is for testing only. I would happily pay for a developer license or something like that.
  - kendallgclark 11y ago
    We sell Developer licenses to developers, indie or not, all the time. Send email to sales@complexible.com! :)
    - vdaniuk 11y ago
      Great news, I will. I guess that option wasn't obvious for me when I was looking over http://stardog.com/#plans http://stardog.com/#plans
pudo 11y ago
The promises they make are really tempting. However, when I experimented with Blazegraph last year, it ended up stuck more often than I could count, whether during bulk import or in answering some ad-hoc SPARQL queries. I can imagine that with extensive tuning, BlazeGraph provides a good database. Just don't expect it to have the polish and convenience of a modern RDBMS or a shiny NoSQL store :)

danbee 11y ago

"Your connection is not secure The owner of www.blazegraph.com has configured their website improperly. To protect your information from being stolen, Firefox has not connected to this website."

beebs 11y ago
We've seen cases where the newer Thawte root certs were not included in the trust chain. That might be the issue.

BenoitP 11y ago

> provides a Scala-based language to write graph and big data analytics and is complementary to the Spark and Hadoop ecosystems It'd be awesome to have Blazegraph as a backend for Spark's Pregel queries. With Tensorflow bindings in place, and the BIDMach/BIDMat libraries, it is very nice seeing Spark getting some serious GPU attention.

beebs 11y ago
We definitely see Spark + Scala + Blazegraph DASL to be a sweet spot for combining the ease of Spark with the GPU performance. Have a submission in to the Hadoop Summit in June on it: https://hadoopsummit.uservoice.com/forums/344955-data-science-analytics-and-spark/suggestions/11664123-the-power-of-spark-the-speed-of-gpus-for-graph-an https://hadoopsummit.uservoice.com/forums/344955-data-scienc...
- rxin 11y ago
  You guys should submit a talk to Spark Summit. Look forward to it.

beebs 11y ago

Got any questions on Blazegraph GPU? Just let us know...

michaelt 11y ago
What's the performance like on data sets that are too big for any single GPU's memory? Can the GPU be used to accelerate shortest-path queries (e.g. dijkstra's algorithm) and if so, where can I read more about how that's achieved?
- beebs 11y ago
  The graph does need to fit into GPU ram. We use graph partitioning for multi-node, Multi-GPU configurations. Dijkstra's algorithm which, as mentioned by Davidson et al. [1], is a "sequential algorithm [that] is poorly suited for parallel architectures like GPUs that require large numbers of parallel threads for efficient execution." Instead, we have variants of the algebraic formulation of the Bellman-Ford algorithm as given in Kepner and Gilbert's book [2]. [1] Andrew A. Davidson, Sean Baxter, Michael Garland, and John D. Owens: "Work-Efficient Parallel GPU Methods for Single-Source Shortest Paths." In Proceedings of the IEEE 28th International Parallel and Distributed Processing Symposium (IPDPS), 2014. http://dx.doi.org/10.1109/IPDPS.2014.45 http://dx.doi.org/10.1109/IPDPS.2014.45 [2] Kepner and Gilbert: "Graph Algorithms in the Language of Linear Algebra."