4 ms·
Hi everyone. Library author here. I worked on this a bunch back in maybe 2009-2010. It's inspired by the work I did with tectonic on selectorgadget. So my c
by fizx 12y ago
Hi everyone. Library author here. I worked on this a bunch back in maybe 2009-2010. It's inspired by the work I did with tectonic on selectorgadget.
So my current thinking on the idea is reflected in https://github.com/fizx/pquery https://github.com/fizx/pquery. PQuery addresses some weaknesses of Parsley by embedding the ideas in Javascript.
(1) Parsley isn't turing-complete, and many web pages are ugly, so you often have to resort to pre/post-processing in some scripting language. I never was able to get sufficient power out of a purely declarative language.
(2) Javascript environments are readily available (even in embedded form), and are more accessible than C.
(3) If your crawler already executes Javascript to render dynamic pages, then running more Javascript in that environment is pretty easy.
I guess I'm a little late to the thread (yay weekends) but I'll answer any questions people may have.
- tectonic 12y agoThose were fun times :) Ruby bindings: https://github.com/fizx/parsley-ruby https://github.com/fizx/parsley-ruby Python bindings: https://github.com/fizx/pyparsley https://github.com/fizx/pyparsley