Streaming parsers may be the exception, but are likely the correct tool to use when handling large inputs becomes a problem. Similarly, XML parsers evolved the

Streaming parsers may be the exception, but are likely the correct tool to use when handling large inputs becomes a problem. Similarly, XML parsers evolved the SAX model (as opposed to DOM, where the document had to fit in memory.) It is often sufficient to rip through the file in an event-based fashion to extract what you want, and throw the rest away. For cases where a single string is truly massive, I would lean towards defining a maximum size to prevent denial of service attacks.