3 ms·

I'm surprised nobody has mentioned WWW::Mechanize - classic perl library [1] or python port of it [2], which is much closer to RoboBrowser than selenium/phantom

by aexaey 10y ago

I'm surprised nobody has mentioned WWW::Mechanize - classic perl library [1] or python port of it [2], which is much closer to RoboBrowser than selenium/phantomjs/horseman. [1] http://search.cpan.org/~ether/WWW-Mechanize-1.75/lib/WWW/Mechanize.pm http://search.cpan.org/~ether/WWW-Mechanize-1.75/lib/WWW/Mec... [2] https://pypi.python.org/pypi/mechanize/ https://pypi.python.org/pypi/mechanize/

Dolores12 10y ago
Mechanize is outdated and python 2 only. We tried it and switched to RoboBrowser.
- grewil 10y ago
  I find WWW::Mechanize::Firefox an interesting alternative. The API is similar to WWW::Mechanize, but it remote controls an instance of Firefox using the MozRepl addon. This means scraping/interacting with pages using JS works fine. http://search.cpan.org/~corion/WWW-Mechanize-Firefox-0.78/lib/WWW/Mechanize/Firefox.pm http://search.cpan.org/~corion/WWW-Mechanize-Firefox-0.78/li...
- jimmaswell 10y ago
  In some significant ways, python 2 is the better language than 3, though.
  - tmerr 10y ago
    In what ways?
    - popey456963 10y ago
      Python 3 fan myself, but there are two things I've heard from colleagues: 1. More modules for Python2 than Python3. A lot of projects are forced to go down to Python2 to allow them to use the modules they want to use. 2. It's what they're familiar with. Generally the older developers like using Python2 because they know all what they're getting, no matter what strings are attached. 3. Better syntax. Apparently some people enjoy the Python2 syntax more than the Python3 one. Not using brackets for print statements seem to be the biggest plus, even though in my opinion it looks more Pythonic.
b3b0p 10y ago
I've used the Perl version a bit and loved it many years ago, but I've also had great success more recently with the Ruby [0] version of Mechanize. [0] https://github.com/sparklemotion/mechanize https://github.com/sparklemotion/mechanize

by aexaey 10y ago

Dolores12 10y ago

Mechanize is outdated and python 2 only. We tried it and switched to RoboBrowser.

grewil 10y ago
I find WWW::Mechanize::Firefox an interesting alternative. The API is similar to WWW::Mechanize, but it remote controls an instance of Firefox using the MozRepl addon. This means scraping/interacting with pages using JS works fine. http://search.cpan.org/~corion/WWW-Mechanize-Firefox-0.78/lib/WWW/Mechanize/Firefox.pm http://search.cpan.org/~corion/WWW-Mechanize-Firefox-0.78/li...
jimmaswell 10y ago
In some significant ways, python 2 is the better language than 3, though.
- tmerr 10y ago
  In what ways?
  - popey456963 10y ago
    Python 3 fan myself, but there are two things I've heard from colleagues: 1. More modules for Python2 than Python3. A lot of projects are forced to go down to Python2 to allow them to use the modules they want to use. 2. It's what they're familiar with. Generally the older developers like using Python2 because they know all what they're getting, no matter what strings are attached. 3. Better syntax. Apparently some people enjoy the Python2 syntax more than the Python3 one. Not using brackets for print statements seem to be the biggest plus, even though in my opinion it looks more Pythonic.

b3b0p 10y ago

I've used the Perl version a bit and loved it many years ago, but I've also had great success more recently with the Ruby [0] version of Mechanize. [0] https://github.com/sparklemotion/mechanize https://github.com/sparklemotion/mechanize