5 ms·
Web crawler
what is the best refrence for studying basic we crawling?
- wlrd 12y agoIf you have an urge to work in python, I would strongly recommend Scrapy. You could also use Beautiful Soup if you wanted. BS would be a little harder to use but is very useful.
- deleted 12y ago[deleted]
- jafingi 12y agoYou should read about screen scraping :-) That's the essence of web crawling.
- kachen 12y agothank u bro.. Mr jafingi do u have any good refrence of screen scraping
- jafingi 12y agoWell, you'd have to understand the fundamentals of DOM: http://www.w3.org/TR/DOM-Level-2-Core/introduction.html http://www.w3.org/TR/DOM-Level-2-Core/introduction.html Also learn about XPath, which is often used as a language to selecting nodes in documents. Scrapy and BeautifulSoup are popular libraries for scraping in Python.
- kachen 12y agothank u bros, actually I am planning to make a livescore app. in android and need online feeds for it.. using API seems limited and I don't know any thing of web crawling...So suggest what should I do.