I would imagine the key to solving this problem effectively is less about parsing DOM and more about recognizing what content on a page is valuable and relevant

I would imagine the key to solving this problem effectively is less about parsing DOM and more about recognizing what content on a page is valuable and relevant. I could see an algorithm merely trying to find the longest prose on a page, then analyzing it for content and then trying to find other content within other elements that are likely relevant. Armed with this and multiple articles from the same source, you should be able to correlate the two to get a smarter parser that can adapt to changing DOM on each site.