3 ms·
Just please consider that XPath needs to store the items in a tree structure first (XML needs to be parsed and loaded into memry) before it can be useful. Runn
by matan_a 14y ago
Just please consider that XPath needs to store the items in a tree structure first (XML needs to be parsed and loaded into memry) before it can be useful. Running that on a large log file would be an interesting performance experiment.
- deno 14y agoNonsense, XPath works very well with streamed content, and there are even implementations of XPath engines for FPGA and GPGPU, all of which have very strict memory limitations. DOM is completely optional, and you only need it for convenience or to observe the modifications, e.g. the way browsers make use of it. Just think how you would compile any XPath expression — it’s very similar to REGEX, only for structured data.
- empthought 14y agoI think you're overstating the capabilities and implementation of common XPath processors here, and silently implying the use of a subset or modified version of XPath as specified. Arbitrary XPath will likely require loading the entire document in the worst case, because it allows navigation to any portion of the tree.
- deno 14y agoNavigation to any portion of the tree does not imply any need to keep the entire document in memory at once. Worst case scenario XPath expression will not yield any results before traversing the entire tree. However any XPath 1.0 (and I imagine 2.0 is no different in that regard) can be compiled into a deterministic state machine (DFA), which only needs to keep tabs on how many elements it has seen and what conditions has been met. What XPath 1.0 specifically doesn’t allow are arbitrary sub-expressions in predicates. Those would be problematic in certain conditions. The only potential issue with performance is when running a large number of XPEs against a single stream. So there are various techniques to remedy that, including merging state machines for branching expression etc.
- empthought 14y agoThere's a following:: axis in XPath which lets you look ahead to arbitrary elements. So the processor needs to load the document into memory, which of course is not streaming.
- deno 14y agoI don’t see how this axis itself is a problem. Could you provide a specific example?
- empthought 14y agoJust /node/node2/following::* in a predicate or whatever. Perhaps you're conflating XPath-as-used with XPath-as-specified? http://msdn.microsoft.com/en-us/library/ms950778.aspx http://msdn.microsoft.com/en-us/library/ms950778.aspx explains that you can't support all of XPath (even 1.0) and process things the way that you describe.
- deno 14y agoRight, they completely eliminate any buffering. That’s very strict, but you’re right. I thought you meant that there’s a subset of XPath that can’t work on anything but DOM.
- empthought 14y agoYeah, that's all. The reason it's relevant is that XPath processors often value conformance over expanding the utility of their system. I didn't want someone going off and using libxslt or xalan expecting it to process things in a memory-efficient manner.