The metrics used in the comparison seems significantly flawed. It compares a reference set of tokens produced by applying the following rules: 1) Remove any so

The metrics used in the comparison seems significantly flawed. It compares a reference set of tokens produced by applying the following rules: 1) Remove any sort of remaining inline html tags 2) Remove all punctuation characters 3) Remove all control characters 4) Remove all non ascii characters (due to unreliable information of the document encoding) 5) Normalize to lowercase 6) Split on whitespace This seems to me a case of measuring what is easy to measure rather than measuring what is right. What would the author think about adding a rule to 'remove all vowels' or 'arbitrarily split words'? Yet he happily removes meaning and context in the form of punctuation and case. If the underlying text extraction algorithms are not similarly handicapped, then one or more of them might be a better standard of measurement than the one the author applies. Rather like measuring the accuracy of an atomic clock by using a rusty stopwatch.