19 ms·

Titan is a new real-time, distributed, transactional graph database that can use either Cassandra or HBase as its distributed data store. Titan 0.3 was stresse

by espeed 14y ago

Titan is a new real-time, distributed, transactional graph database that can use either Cassandra or HBase as its distributed data store. Titan 0.3 was stressed tested with Cassandra at 120 billion edges and is capable of loading 1.2 million edges per second on a 16 machine hi1.4xl cluster (https://twitter.com/aureliusgraphs/status/316255164719828992 https://twitter.com/aureliusgraphs/status/316255164719828992). This release provides a complete performance-driven redesign of many core components, and the primary new feature is advanced indexing. Here are the new indexing features: * Geo: Search for elements using shape primitives within a 2D plane. * Full-text: Search elements for matching string and text properties. * Numeric range: Search for elements with numeric property values using intervals. * Edge: Edges can be indexed as well as vertices. See http://thinkaurelius.com/news/ http://thinkaurelius.com/news/

erichocean 14y ago
Hey espeed, how well would the SybilGuard[0] algorithm run on Titan? Right now, we're evaluating Twitter's in-memory graph library (Cassovary[1]), but that obviously requires a big machine..., and we already use Cassandra elsewhere, so would prefer to stay with that. Up to, say, 5 second response times on large graphs would be acceptable. [0] http://www.math.cmu.edu/~adf/research/SybilGuard.pdf http://www.math.cmu.edu/~adf/research/SybilGuard.pdf [1] https://github.com/twitter/cassovary https://github.com/twitter/cassovary
- espeed 14y ago
  Titan is one piece of the Aurelius Graph Cluster (http://thinkaurelius.com/subscription/ http://thinkaurelius.com/subscription/). 1. Titan is the OLTP piece, and it's very fast at running local-rank algorithms (http://markorodriguez.com/2011/03/30/global-vs-local-graph-ranking/ http://markorodriguez.com/2011/03/30/global-vs-local-graph-r...). 2. Faunus is an OLAP graph-analytics engine that integrates Titan with Hadoop for global analysis of Titan graphs. Graphs are analyzed using a MapReduce implementation of the Gremlin graph traversal language. General use-cases include computing graph derivations/transformations and global graph statistics. You can then feed the global-algo results back into Titan. 3. Fulgora is an in-memory, compression-based, transaction-less OLAP graph processor capable of storing billions of edges within the memory confines of a single machine. Fulgora is optimized for the execution of massively threaded, global graph algorithms. It will come out later this year, and you can connect it to Faunus or feed it directly from Titan. If you can construct a local SybilGuard algo, you can run it in Titan and get an immediate response. Otherwise, for global-graph algos, you would feed Faunus from Titan directly and query Faunus' in-memory graph. There are also things in the works that will blur these distinctions. More details to come later this month in a series of blog posts -- stay tuned. Marko or Matthias will have more insight on how to best run a SybilGuard-type algo. Right now they're about to jump on a flight to Austin for Data Day Texas (http://datadaytexas.com/ http://datadaytexas.com/), but I'm sure they'll respond when they have a free moment. See Marko's YOW! interview (http://channel9.msdn.com/posts/YOW-2012-Marko-Rodriguez-Graph-Systems-and-Databases http://channel9.msdn.com/posts/YOW-2012-Marko-Rodriguez-Grap...) and Matthias' Titan/Cassandra talk (http://www.youtube.com/watch?v=ZkAYA4Kd8JE http://www.youtube.com/watch?v=ZkAYA4Kd8JE) for more details on the architecture.
  - mhluongo 14y ago
    Very exciting stuff! Looking forward to seeing you guys at DDT.
  - espeed 14y ago
    CORRECTION -- that should say: "Otherwise, for global-graph algos, you would feed Fulgora from Titan directly and query Fulgora's in-memory graph" (sorry if I confused anyone).
  - erichocean 14y ago
    Thanks espeed, the links were especially helpful. I'm looking forward to Fulgora, sounds great!
    - espeed 14y ago
      No problem. I sent a tweet to @erichocean -- is that you? If you want, let's chat about algo options next week when everyone is back in town.
anigbrowl 14y ago
Excellent documentation - this is what a 'getting started' page should look like. All projects should have this depth of introductory material.

by espeed 14y ago

erichocean 14y ago

Hey espeed, how well would the SybilGuard[0] algorithm run on Titan? Right now, we're evaluating Twitter's in-memory graph library (Cassovary[1]), but that obviously requires a big machine..., and we already use Cassandra elsewhere, so would prefer to stay with that. Up to, say, 5 second response times on large graphs would be acceptable. [0] http://www.math.cmu.edu/~adf/research/SybilGuard.pdf http://www.math.cmu.edu/~adf/research/SybilGuard.pdf [1] https://github.com/twitter/cassovary https://github.com/twitter/cassovary

espeed 14y ago
Titan is one piece of the Aurelius Graph Cluster (http://thinkaurelius.com/subscription/ http://thinkaurelius.com/subscription/). 1. Titan is the OLTP piece, and it's very fast at running local-rank algorithms (http://markorodriguez.com/2011/03/30/global-vs-local-graph-ranking/ http://markorodriguez.com/2011/03/30/global-vs-local-graph-r...). 2. Faunus is an OLAP graph-analytics engine that integrates Titan with Hadoop for global analysis of Titan graphs. Graphs are analyzed using a MapReduce implementation of the Gremlin graph traversal language. General use-cases include computing graph derivations/transformations and global graph statistics. You can then feed the global-algo results back into Titan. 3. Fulgora is an in-memory, compression-based, transaction-less OLAP graph processor capable of storing billions of edges within the memory confines of a single machine. Fulgora is optimized for the execution of massively threaded, global graph algorithms. It will come out later this year, and you can connect it to Faunus or feed it directly from Titan. If you can construct a local SybilGuard algo, you can run it in Titan and get an immediate response. Otherwise, for global-graph algos, you would feed Faunus from Titan directly and query Faunus' in-memory graph. There are also things in the works that will blur these distinctions. More details to come later this month in a series of blog posts -- stay tuned. Marko or Matthias will have more insight on how to best run a SybilGuard-type algo. Right now they're about to jump on a flight to Austin for Data Day Texas (http://datadaytexas.com/ http://datadaytexas.com/), but I'm sure they'll respond when they have a free moment. See Marko's YOW! interview (http://channel9.msdn.com/posts/YOW-2012-Marko-Rodriguez-Graph-Systems-and-Databases http://channel9.msdn.com/posts/YOW-2012-Marko-Rodriguez-Grap...) and Matthias' Titan/Cassandra talk (http://www.youtube.com/watch?v=ZkAYA4Kd8JE http://www.youtube.com/watch?v=ZkAYA4Kd8JE) for more details on the architecture.
- mhluongo 14y ago
  Very exciting stuff! Looking forward to seeing you guys at DDT.
- espeed 14y ago
  CORRECTION -- that should say: "Otherwise, for global-graph algos, you would feed Fulgora from Titan directly and query Fulgora's in-memory graph" (sorry if I confused anyone).
- erichocean 14y ago
  Thanks espeed, the links were especially helpful. I'm looking forward to Fulgora, sounds great!
  - espeed 14y ago
    No problem. I sent a tweet to @erichocean -- is that you? If you want, let's chat about algo options next week when everyone is back in town.

anigbrowl 14y ago

Excellent documentation - this is what a 'getting started' page should look like. All projects should have this depth of introductory material.