3 ms·
While BeautifulSoup is great, lxml + xpath really is the way to go. XPath is a W3C standard and works cross language and even in the browser. If you need a an
by turtlebits 5y ago
While BeautifulSoup is great, lxml + xpath really is the way to go. XPath is a W3C standard and works cross language and even in the browser.
If you need a an quick way to scrape javascript generated content, you can just open your browser console and use `document.evaluate` with an XPath query.
- udev 5y agoCan you please elaborate? Are you thinking more about the performance, or code maintainability? One thing that is difficult is updating the BeautifulSoup code every time a website changes design/layout/etc.
- turtlebits 5y agoXPath is like SQL, learn it once. Anything you write with Beautifulsoup will not translate to any other language or library.