Thanks for posting that, it is exactly what I need. I want to scrape a diverse selection of web pages for 'the gist' of the main content purely to put into a s

Thanks for posting that, it is exactly what I need. I want to scrape a diverse selection of web pages for 'the gist' of the main content purely to put into a solr search engine. At the moment I am doing surprisingly well just going on the meta descriptions - in essence meta descriptions should be what I am after - but meaningfully scraped content is the ideal and what I need if no meta description available. I am sure you have lots of grand plans for this, however, one thing that I wish I could do is search through my web history, at a content level. For instance, earlier today I found some code was no longer working properly. I could remember bits of the tutorial I needed first time around but finding it again was a chore (on a narrow subject then every search result from Google is purple/visited, so you can't immediately identify the link). If I could have done some 'search my previous browsing history' then I could have got there a lot quicker. In this 'imaginary search' only the pages that are kept open and scrolled through get 'internally archived'. There could also be snapshots of the pages in question in the results as an aid to memory. Another thing I would like to see done with this type of tool is a sensible spell check. For decades we have had spell checks in text entry areas but never on a finished web page. A little widget to show the words with problem spellings with wavy underlining that would work on any web page would save no end of woes for people that have to proof read things online. If it also highlighted sentences too long for good copy that would be very useful too. Can these sorts of applications be built on what you have here? P.S. Funny that you wrote this 186 days ago: https://news.ycombinator.com/item?id=6581317 https://news.ycombinator.com/item?id=6581317 Times have changed...