4 ms·
It may not use the MIGHTY POWER OF JAVASCRIPT, but BeautifulSoup is a best-of-breed real-world HTML parser. Not just in the API, but in the verification that it
by jerf 16y ago
It may not use the MIGHTY POWER OF JAVASCRIPT, but BeautifulSoup is a best-of-breed real-world HTML parser. Not just in the API, but in the verification that its parsing algorithm is effective against HTML found on real web sites. I find it unlikely that jsdom is actually significantly better. Does that code really look like it's going to be significantly improved with jQuery?
titles = [x for x in soup.findAll('td','title') if x.findChildren()][:-1]
packs a lot of punch.
- meatmanek 16y agoThe ability to use CSS/jQuery selectors is really nice, though; In order to find all <td>s whose parents have a class "blah", you have to use list comprehension: tds = [td for td in soup.findAll('td') if td.parent.get('class') == 'blah'] In jQuery, this is more compactly written: tds = $('.blah > td') And if you just want to look for <td>s somewhere within a .blah element, you can use tds = $('.blah td') This is a lot less clear in BeautifulSoup: tds = [td for td in soup.findAll('td') if td.findParents(attrs={'class': 'blah'})] (If there are better ways to write this BeautifulSoup code, please let me know) Selectors have some other benefits too - you can just go to the CSS file and grab the selector that matches what you want, and you can be reasonably sure it'll work in most cases.
- joshu 16y agoLxml in python lets you use CSS selectors.
- cdr 16y agoMy impression was that BeautifulSoup is actually getting very long in the tooth - I didn't use it much, but I have used a lot of Scrapy, a beautiful framework that completely supplants BeautifulSoup for scraping.