Handling invalid syntax is another can of worms! It would make all parts of the tokenizer much more complex, not just comments. In the case of XHTML, a parser

Handling invalid syntax is another can of worms! It would make all parts of the tokenizer much more complex, not just comments. In the case of XHTML, a parser is supposed to reject any document which is not well formed. HTML parsers typically try to "gracefully recover" from all syntax errors, but this is a crazy complex algorithm.