4 ms·

Ruby has a library named scrubyt http://scrubyt.org/ http://scrubyt.org/

• astrofinch 16y ago
  Python has http://www.crummy.com/software/BeautifulSoup/ http://www.crummy.com/software/BeautifulSoup/
  • pyre 16y ago
    How relevant are these posts now? http://blog.ianbicking.org/2008/12/10/lxml-an-underappreciated-web-scraping-library/ http://blog.ianbicking.org/2008/12/10/lxml-an-underappreciat... http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/ http://blog.ianbicking.org/2008/03/30/python-html-parser-per...
    • iamelgringo 16y ago
      Very. I recently switched from BeautifulSoup to lxml. It's tons faster, and I like it quite a bit more than BeautifulSoup. The documentation isn't quite as good, and it's a bit of a pain to install, but if you have to do a bunch of scraping, I'd check it out.
      • evilduck 16y ago
        What pain? I've only used lxml for a project on Windows, but there, it was as simple as installing setuptools and then 'easy-install lxml'.
        • pyre 16y ago
          I think the 'pain' comes when there are issues with the c library that lxml binds to.
  • evilduck 16y ago
    I was under the impression BeautifulSoup is no longer actively maintained. (Edit: Which makes me sad, because I loved their documentation writing style)
    • astrofinch 16y ago
      No idea.
    • krakensden 16y ago
      It's not :(
  • jhy 16y ago
    I've been working on a Java HTML parser: http://jsoup.org/ http://jsoup.org/ It supports CSS selectors, jquery-like DOM traversal and set manipulation, etc.
  • andrix 16y ago
    Python has Scrapy (http://scrapy.org http://scrapy.org), quoting: "Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing."