3 ms·

Streaming parsers are key when dealing with XML files this big. Used to have a C# parser that would parse about 1 TB of XML per day the biggest files were > 200

by fleitz 14y ago

Streaming parsers are key when dealing with XML files this big. Used to have a C# parser that would parse about 1 TB of XML per day the biggest files were > 200GB. It was impossible with out rewriting everything to use a SAX style parser.

barrkel 14y ago
SAX style (parsing library callbacks) is not your only option; you can also use an iterator style (i.e. something like XmlReader in C#).
- fleitz 14y ago
  Oops, mistook SAX for iterator, I really prefer XmlReader to SAX style.
- masklinn 14y ago
  > you can also use an iterator style (i.e. something like XmlReader in C#). This category is generally called "pull parsers" (as opposed to SAX-style event-driven parsers)
Jimmie 14y ago
I'm curious, what data did the XML files contain?