4 ms·

I prefer to use html parsers for such problems, such as beautifulsoup in python. I used xpath in the past but the ending code wasn't that much shorter then a mo

by d0m 13y ago

I prefer to use html parsers for such problems, such as beautifulsoup in python. I used xpath in the past but the ending code wasn't that much shorter then a more verbose version based on beautifulsoup. And, for someone looking at the code, the beautiful version makes so much more sense.. Xpath also feels like a big regex expression that magically works. I'm not saying it's not useful. Actually, I believe that if you only have one use-case, then using xpath might be overkill because of all the added-complexity of maintaining a new library/technology/ideology. But if it's the sort of domain that xpath would be useful more than once, then sure use it.

TylerE 13y ago
The thing about xpath is that it runs in highly efficient native code. When iterating over a 100MB+ file it makes an _immense_ difference. It's conceptually the difference between: count = 0 for row in db.query("select id from <table>"): count += 1 and db.query("select count(*) from <table>") I also don't find it at all confusing, you just have to understand the tree nature of XML.
- d0m 13y ago
  Oh no, you've used the performance argument against me ;-) Obviously, when performance issues are on the line, you often need to trade simplicity and maintainability.
  - TylerE 13y ago
    I'm not sure how using a platform specific library is more maintainable than an open standard with myriad implementations.
thangalin 13y ago
http://www.w3.org/Tools/HTML-XML-utils/man1/hxselect.html http://www.w3.org/Tools/HTML-XML-utils/man1/hxselect.html "hxselect - extract elements that match a (CSS) selector" CSS is often a simpler way to extract data. http://www.w3.org/Tools/HTML-XML-utils/ http://www.w3.org/Tools/HTML-XML-utils/
pyre 13y ago
Or you could use lxml in Python, which allows you to mix XPath and CSS Selectors. E.g.: from lxml import html doc = html.fromstring('<html><body><p class="text"></p></body></html>') doc.xpath('//p[@class = "text"]') == doc.cssselect('p.text') # or doc = html.fromstring('<html><body><p>1</p><p>2</p><p>3</p></body></html>') doc.xpath('//p[2]')[0] == doc.cssselect('p')[1] # Note: My only annoyance is that .xpath() always returns a list, even # when you know that it will return only a single item.

by d0m 13y ago

TylerE 13y ago

The thing about xpath is that it runs in highly efficient native code. When iterating over a 100MB+ file it makes an _immense_ difference. It's conceptually the difference between: count = 0 for row in db.query("select id from <table>"): count += 1 and db.query("select count(*) from <table>") I also don't find it at all confusing, you just have to understand the tree nature of XML.

d0m 13y ago
Oh no, you've used the performance argument against me ;-) Obviously, when performance issues are on the line, you often need to trade simplicity and maintainability.
- TylerE 13y ago
  I'm not sure how using a platform specific library is more maintainable than an open standard with myriad implementations.

thangalin 13y ago

http://www.w3.org/Tools/HTML-XML-utils/man1/hxselect.html http://www.w3.org/Tools/HTML-XML-utils/man1/hxselect.html "hxselect - extract elements that match a (CSS) selector" CSS is often a simpler way to extract data. http://www.w3.org/Tools/HTML-XML-utils/ http://www.w3.org/Tools/HTML-XML-utils/

pyre 13y ago

Or you could use lxml in Python, which allows you to mix XPath and CSS Selectors. E.g.: from lxml import html doc = html.fromstring('<html><body><p class="text"></p></body></html>') doc.xpath('//p[@class = "text"]') == doc.cssselect('p.text') # or doc = html.fromstring('<html><body><p>1</p><p>2</p><p>3</p></body></html>') doc.xpath('//p[2]')[0] == doc.cssselect('p')[1] # Note: My only annoyance is that .xpath() always returns a list, even # when you know that it will return only a single item.