For the home page, I request the HTML from HN, process it, and create a collection of elements that represent a HN news item: URL, Title, # of comments, ID, Pos

For the home page, I request the HTML from HN, process it, and create a collection of elements that represent a HN news item: URL, Title, # of comments, ID, Posted By, Date, and so on. Then use that to generate the new HTML. I cache that for 200 seconds and store a copy in the event that I can't read the HTML from HN. Works just about the same way for the comments , except it adds some additional functionality like collapsing comments. With the nature of comments being nested, a recursive function worked really well on rendering the output. The article text functionality uses the algorithm from arc 90's readability project, which is in javascript. I used the main idea behind their readability algorithm for this, except mine runs server side (C#) where theirs is all done on the client.