4 ms·The lexer is the bit that breaks up the string into tokens (words in this case).by olliesaunders 16y agoThe lexer is the bit that breaks up the string into tokens (words in this case).