4 ms·
> but had fun doing it Well, that's more than a lot of people can say about certain projects! Want to give us a run-down of how you put it together?
by thinkalone 16y ago
> but had fun doing it
Well, that's more than a lot of people can say about certain projects!
Want to give us a run-down of how you put it together?
- cnlwsu 16y agoDefinitely, it was actually pretty simple once I figured out what I was going to do. I started from the link graph put together by Henry Haselgrove (http://users.on.net/~henry/home/wikipedia.htm http://users.on.net/~henry/home/wikipedia.htm) that I found when looking through the EC2 public datasets. I then had a few easy steps. 1) flip the link graph from outgoing to incoming, so from any page I can see what links to it. 2) I found all the distances and paths iteratively by exploding out from Adolf_Hitlers page. http://www.johnandcailin.com/blog/cailin/breadth-first-graph-search-using-iterative-map-reduce-algorithm http://www.johnandcailin.com/blog/cailin/breadth-first-graph... and blogs like it were very helpful. 3) loaded the data into a large binary file that I divided into indexed parts that I compressed and uploaded to appengine to extract and load into bigtable (this took the most amount of time! both to run and to write the code to make it work) 4) ?? 5) profit
- kenjackson 16y agoCool. How long did it take and how big is it? Also, what's the longest path?