3 ms·
Other posters have replied giving a back of the envelope as to feasibility, so from the other end: The PageRank implementation on Dato's GraphLab Create when r
by Smerity 11y ago
Other posters have replied giving a back of the envelope as to feasibility, so from the other end:
The PageRank implementation on Dato's GraphLab Create when run on the Web Data Commons Hyperlink Graph (128 billion edges) does 3 billion edges a second on 16 nodes, which is 187 million edges per second per machine.
Given that communication overhead quickly becomes an issue for most of these systems and their graph is dealing with more edges, 28 million edges per second per machine seems quite reasonable.
[1]: https://twitter.com/CommonCrawl/status/623615774909857792 https://twitter.com/CommonCrawl/status/623615774909857792