I did something very similar with XML in Ruby. All the libraries are either event-based (SAX, which is awful for actual parsing code) or snarf the whole documen

I did something very similar with XML in Ruby. All the libraries are either event-based (SAX, which is awful for actual parsing code) or snarf the whole document into RAM. I wrote a simple SAX wrapper [1] that applies a (so far, extremely simplified) subset of XPath to the stream, then parses each match into a DOM using Nokogiri. Works exceedingly well, though it's not faster than reading everything into RAM. [1] https://github.com/t11e/nokogiri-simple-streaming https://github.com/t11e/nokogiri-simple-streaming