3 ms·
How fast is compare to lxml with the HTML parser ? I know that lxml won't parse as many cases as this module, but in many cases lxml can do the job.
by dalf 9y ago
How fast is compare to lxml with the HTML parser ?
I know that lxml won't parse as many cases as this module, but in many cases lxml can do the job.
- aumerle 9y agoI have not tested it, but it is probably slower. The advantage of this is that is follows the HTML 5 parsing spec, so it gives you the same results as a browser when parsing malformed HTML
- Paul-ish 9y agoI have found the lxml API in python leaks a considerable amount of memory, but that was while going through a large amount of data though (Wikipedia dump). This is in contrast to a handful of pages. But that was a while ago, it may be different now.
- Buttons840 9y agoWhat do you mean by "leak"? I wrote a scraper that used lxml and a single instance/process ran for month on end in production without any issues. Lxml may have used more memory than some alternatives, but I don't think it was "leaking" memory.
- Paul-ish 9y agoThat is likely true. My experience with it was in 2011/2012 ish. Their FAQ[1] claims memory leaks have been fixed over time. * http://lxml.de/FAQ.html http://lxml.de/FAQ.html