Ghost.py seems promising but is painful to work with. Installing the dependencies took about an hour (compiling PyQt itself took about 25 minutes). When we did

Ghost.py seems promising but is painful to work with. Installing the dependencies took about an hour (compiling PyQt itself took about 25 minutes). When we did finally get it working, PyQt would crash after the third or fourth scrape and would take between 15 to 30 seconds per scrape. Where we used it: https://github.com/createch/PriceChecker.py/blob/master/pychecker/scraper.py https://github.com/createch/PriceChecker.py/blob/master/pych.... My advice to anybody looking to do headless webkit in Python: Don't use Ghost. Try out PhantomPy and try using the Selenium webdriver for Phantom. We switched over the Selenium webdriver for PhantomJS and found it to be much more stable and fast. See this SO answer by another Ghost.py user who gave up: http://stackoverflow.com/a/15699761/854025 http://stackoverflow.com/a/15699761/854025. Where we used the Selenium driver: https://github.com/createch/PriceChecker.py/blob/phantom/pychecker/scraper2.py https://github.com/createch/PriceChecker.py/blob/phantom/pyc....