While I don't have a ton of experience when it comes to writing lexers and parsers, I have in fact written a couple small ones based on that specific Rob Pike t

While I don't have a ton of experience when it comes to writing lexers and parsers, I have in fact written a couple small ones based on that specific Rob Pike talk. I honestly found the talk itself enlightening and have recommended it to many people. I regularly use one of them to parse multi-gigabyte files for work and have been mildly disappointed with the results. I'd be curious the performance implications of straight up serializing the process with maybe an RWMutex wrapping the token handoff to see what amount of that is the channel itself? One of the lexers/parser based on the talk - https://github.com/donatj/sqlread https://github.com/donatj/sqlread