That speedup is not unconditional. It only occurs in portions of the input string that don't trigger any state transitions. And the length of those portions has

That speedup is not unconditional. It only occurs in portions of the input string that don't trigger any state transitions. And the length of those portions has to be some reasonable multiple of the size of your vector register. What constitutes a state transition will depend highly on the structure of your input string and the information you wish to extract from it. As an example, if I want to build a nested array/map of things from this input JSON: {"a": "bcd", "ef": [{},{},{"a":12,"b":100},{"a":12,"b":100},{"a":12,"b":100}, ...]} SIMD will not provide an appreciable performance boost because the data type changes every few characters. There's no string or whitespace or number in that JSON that is large enough to speed up scanning.