OK, here's the script that drives the process. echo Retrieving index curl http://www.paulgraham.com/articles.html > data0 echo Extracting URLs

OK, here's the script that drives the process. echo Retrieving index curl http://www.paulgraham.com/articles.html > data0 echo Extracting URLs grep -ho "<a href[^>]*>[^<]*<.a>" data0 | grep -v http | grep -v RSS > data1 ComputeRankings takes the list of essays, extracts outgoing links to create a digraph, and then computes the Page-Rank of each page. CreatePGER takes the results and an HTML template and glues them together to create the rankings page. echo Creating rankings page ./ComputeRankings.py > ComputeRankings.out ./Create_PGER.py > PaulGrahamEssaysRanking MakeGraph outputs dot files for the giant component and the "other nodes" graph. echo Creating DOT file ./MakeGraph.py > links0.dot Then I use neato to layout the graphs. In each case I run it a few hundred times and pick the result that has the largest boxes. That means it's the most compact output, and seems to be a good heuristic. echo Create Giant.png ./LayoutGraph Giant 11,40 echo Create Other.png ./LayoutGraph Other 8,5 Finally I create the HTML you see using an HTML template. echo Create page with map ./Create_PGE.py > PaulGrahamEssays There's a small lie in this. The web site is actually a statically generated "wiki". When it was first devised I didn't have the facility to run scripts on my host. I generate the pages as plain text with some mark-down, then off-line generate the entire site. Then I upload the parts that have changed. If you try to edit then your suggested new version gets emailed to me, where it goes through an aggressive spam filter. Then it sits in my inbox for me to decide if it's a good change. If so, I trigger a refresh. Some people have passwords that trigger the refresh automatically, without my intervention, so it really does work as a limited access wiki. Er, right. Is that what you wanted to know?