4 ms·
I do my scrapping with BeautifulSoup from Python, actually from an iPython shell. With a urllib2 opener you could easily handle cookies and UserAgent pretty eas
by babo 18y ago
I do my scrapping with BeautifulSoup from Python, actually from an iPython shell. With a urllib2 opener you could easily handle cookies and UserAgent pretty easy, the later is also important for some sites.
- geoscripting 18y agoThere's also mechanize for python. It can be found here : http://wwwsearch.sourceforge.net/mechanize/ http://wwwsearch.sourceforge.net/mechanize/ . It handles cookies by default, and is a pretty good tool.