4 ms·
Is mechanize (http://wwwsearch.sourceforge.net/mechanize/ http://wwwsearch.sourceforge.net/mechanize/) considered outdated or convoluted? It's what I've used fo
by tsumnia 15y ago
Is mechanize (http://wwwsearch.sourceforge.net/mechanize/ http://wwwsearch.sourceforge.net/mechanize/) considered outdated or convoluted? It's what I've used for my scrapings.
Also, how well do these other scrapers handle Javascript? I've had to abandon some scrapes from ASP pages because they wouldn't properly handle it.
- danohuiginn 15y agoit's a different tool. Mechanize concentrates on navigating pages and downloading pages, following links, handling cookies etc. BeautifulSoup and lxml parse information out of the html. There's some overlap, but not much. I have tended to use BeautifulSoup and mechanize together. As mentioned above, BeautifulSoup is no longer being actively maintained, and I'd recommend starting with lxml in most cases. I'm still using BeautifulSoup mainly because I have most of the package memorized.
- tsumnia 15y agoThanks, I use the same combination, but haven't needed to make any new parses in a while. I'll read up on lxml for the next time.
- wesleyzhao 15y agoIs there any other tool that easily help with JS parsing?? This would actually be extremely useful since there are so many JavaScript generated pages.