3 ms·
That sounds like an awesome tool! If you don't mind me asking, is it publicly available to view anywhere? What language is it in? To second another comment, I
by hello_newman 9y ago
That sounds like an awesome tool! If you don't mind me asking, is it publicly available to view anywhere? What language is it in?
To second another comment, I am also working on scraping but for articles. In this particular example I am trying to figure out how to get the scraper to start on the first headline all the way down to the bottom, and not touch anything else. Any tips for how to get it to find the first heading on a page?
- assafmo 9y agoI would think it depends on the site... maybe some special html characteristics around the first headline? I usually use lynx, which strips down the html and only outputs the text. News site tend to have the same structure for every article, so after passing the html to lynx, the headline for a news site article will probably be around line 5-6.