4 ms·

Pregel: Large-scale graph computing at Google

36 points by jashmenn 16y ago

jashmenn 16y ago
Does anyone have any more information about this? Is it built on top of MapReduce or is it totally separate from the ground up?
- buro9 16y ago
  And is the paper from last year available anywhere? Are examples of implementations available?
  - strebler 16y ago
    Apprently this is the paper: http://projects.will.madstones.com/thesis/papers/Malewicz%20et%20al.%20-%20Pregel:%20a%20system%20for%20large-scale%20graph%20processing.pdf http://projects.will.madstones.com/thesis/papers/Malewicz%20... But it's just 1 page, a bit of a letdown...
  - MLnick 16y ago
    http://portal.acm.org/citation.cfm?id=1582716.1582723 http://portal.acm.org/citation.cfm?id=1582716.1582723 only paper I found. It's only a page though!
    - abeppu 16y ago
      But it's on the accepted papers list for sigmod/pods 2010, so with luck in a month or so we'll know more. http://j.mp/aPJuyA http://j.mp/aPJuyA
- jashmenn 16y ago
  I just checked with my coworker Marko (author of Gremlin [1]) and he told me that no-one really has much info on this. Though he said that Apache Hama [2] + Angrapa [3] is a supposed to be a clone. I think Hama is built on MR so maybe Google's version is as well? [1] http://github.com/tinkerpop/gremlin http://github.com/tinkerpop/gremlin [2] http://incubator.apache.org/hama/ http://incubator.apache.org/hama/ [3] http://wiki.apache.org/hama/GraphPackage http://wiki.apache.org/hama/GraphPackage
- helwr 16y ago
  http://www.royans.net/arch/pregel-googles-other-data-processing-infrastructure/ http://www.royans.net/arch/pregel-googles-other-data-process... http://blog.udanax.org/2009/08/inference-anatomy-of-google-pregel.html http://blog.udanax.org/2009/08/inference-anatomy-of-google-p... http://portal.acm.org/citation.cfm?id=1073981 http://portal.acm.org/citation.cfm?id=1073981 also check out PIs publications and his thesis http://research.google.com/pubs/author145.html http://research.google.com/pubs/author145.html The lead author will also be giving a presentation on Pregel at SIGMOD 2010: http://www.sigmod2010.org/research_list.shtml http://www.sigmod2010.org/research_list.shtml
- deleted 16y ago
  [deleted]
iamwil 16y ago
"Implementing PageRank, for example, takes only about 15 lines of code." Interesting. I wonder what it looks like.
- sp332 16y ago
  It's a white powdery substance... oh, code, never mind.
nysauhem 16y ago
I wonder if this can be considered an NP-complete oracle.

36 points by jashmenn 16y ago

Does anyone have any more information about this? Is it built on top of MapReduce or is it totally separate from the ground up?

buro9 16y ago
And is the paper from last year available anywhere? Are examples of implementations available?
- strebler 16y ago
  Apprently this is the paper: http://projects.will.madstones.com/thesis/papers/Malewicz%20et%20al.%20-%20Pregel:%20a%20system%20for%20large-scale%20graph%20processing.pdf http://projects.will.madstones.com/thesis/papers/Malewicz%20... But it's just 1 page, a bit of a letdown...
- MLnick 16y ago
  http://portal.acm.org/citation.cfm?id=1582716.1582723 http://portal.acm.org/citation.cfm?id=1582716.1582723 only paper I found. It's only a page though!
  - abeppu 16y ago
    But it's on the accepted papers list for sigmod/pods 2010, so with luck in a month or so we'll know more. http://j.mp/aPJuyA http://j.mp/aPJuyA
jashmenn 16y ago
I just checked with my coworker Marko (author of Gremlin [1]) and he told me that no-one really has much info on this. Though he said that Apache Hama [2] + Angrapa [3] is a supposed to be a clone. I think Hama is built on MR so maybe Google's version is as well? [1] http://github.com/tinkerpop/gremlin http://github.com/tinkerpop/gremlin [2] http://incubator.apache.org/hama/ http://incubator.apache.org/hama/ [3] http://wiki.apache.org/hama/GraphPackage http://wiki.apache.org/hama/GraphPackage
helwr 16y ago
http://www.royans.net/arch/pregel-googles-other-data-processing-infrastructure/ http://www.royans.net/arch/pregel-googles-other-data-process... http://blog.udanax.org/2009/08/inference-anatomy-of-google-pregel.html http://blog.udanax.org/2009/08/inference-anatomy-of-google-p... http://portal.acm.org/citation.cfm?id=1073981 http://portal.acm.org/citation.cfm?id=1073981 also check out PIs publications and his thesis http://research.google.com/pubs/author145.html http://research.google.com/pubs/author145.html The lead author will also be giving a presentation on Pregel at SIGMOD 2010: http://www.sigmod2010.org/research_list.shtml http://www.sigmod2010.org/research_list.shtml
deleted 16y ago
[deleted]

iamwil 16y ago

"Implementing PageRank, for example, takes only about 15 lines of code." Interesting. I wonder what it looks like.

sp332 16y ago
It's a white powdery substance... oh, code, never mind.

nysauhem 16y ago

I wonder if this can be considered an NP-complete oracle.