13 ms·
PHP Simple HTML DOM Parser
- grumpycanuck 17y agoAs someone who works with XML all day long, I am wondering why people don't use SimpleXML in PHP 5+ for this sort of thing too.
- tetsuo13 17y agoSimpleXML will choke on invalid HTML/XML while this library claims to support it.
- mildweed 17y agoThis is a good one to be sure. Also, check out QueryPath (http://querypath.org/ http://querypath.org/).
- kylemathews 17y agoI used QueryPath for a recent project and really liked it. Its selectors are almost identical to jQuery's so the learning curve was shallow. Two thing QueryPath does that this library doesn't seem to is work w/ XML and allow for chainable method calls. See this IBM Developer Work's article http://www.ibm.com/developerworks/opensource/library/os-php-querypath/ http://www.ibm.com/developerworks/opensource/library/os-php-...
- gorm 17y agohttp://code.google.com/p/phpquery/ http://code.google.com/p/phpquery/ This also seems interesting.
- encoderer 17y agoI've been using this for some serious scraping recently... dozens of threads scraping for 5, 6 days at a time. Very good application, but watch out for memory leaks. Leaks like a sieve when you miss a ->clear() call in your destructors if you're not using the circular reference detection feature in the 5.3 garbage collector.
- yannis 17y agoAnother way to get to the DOM, especially if you are using tidy to clean-up your HTML is tidy itself http://php.net/manual/en/book.tidy.php http://php.net/manual/en/book.tidy.php It is not very hard to get elements using tidy, but you need to write your own functions. However, is almost bullet-proof as it can correct for malformed HTML which some of the other libaries don't.
- dshah 17y agoI'm using Tidy now, and it's been working pretty well. What I do is use Tidy to convert mal-formed HTML to XHTML and then use the Simple XML methods for my parsing. Anyone know if this new library is better than that approach?
- danw 17y agoI've used this before. It's got a lovely api but uses far too much memory.
- larrykubin 17y agoFor Python folks, check out pyquery (http://www.pyquery.org/ http://www.pyquery.org/). It's really handy.
- crux_ 17y agoSince folks here seem to be in the know, if I were after "parsing as close to the browser as possible", ideally in Java/Scala or Ruby or Python (sorry PHP!) -- any recommendations? I've done scraping (e.g. w/ BeautifulSoup) but haven't looked to see how true the parses are to what IE/FF/WebKit would produce. (On my list of things to look into: html5lib -- http://code.google.com/p/html5lib/ http://code.google.com/p/html5lib/ ... is it any good?)
- tremendo 17y agofor Ruby there's Hpricot and Nokogiri. Now I must admit not understanding what "parsing as close to the browser as possible" would mean. These parsers would not be for displaying the HTML, they're not rendering engines like those in browsers, but will help you navigate the DOM of a (X)(HT)ML document programatically. Surely I'm missing your meaning.
- crux_ 17y agoWhat I'm aiming at would mean "given this lump of (malformed) HTML, what DOM would a browser give me?" Maybe Hpricot, BeautifulSoup, et al are already there, but I don't know. :)
- fizx 17y agoRun headless Firefox+xvfb+Selenium. Selenium has java, ruby, and python clients. This is working really well in production. You might also have luck with AppleScript/RubyCocoa/MacRuby+Safari. I've tried a couple other options like HTMLUnit (which google uses with GWT), and Mozilla Java Html Parser, which Dapper.net uses. I couldn't get them running, but YMMV.
- vinhboy 17y agoyou guys read my mind.. i've been needing one of these.