5 ms·
You might try looping your tokenizer in its own goroutine and passing tokens along a channel. That separates emitting a token from the loop/return structure of
by zmj 14y ago
You might try looping your tokenizer in its own goroutine and passing tokens along a channel. That separates emitting a token from the loop/return structure of next_token.
Edit: check out this example http://blog.golang.org/2011/09/two-go-talks-lexical-scanning-in-go-and.html?m=1 http://blog.golang.org/2011/09/two-go-talks-lexical-scanning...
- wonderzombie 14y agoHere's the source if you just want to dive in: http://golang.org/src/pkg/text/template/parse/lex.go http://golang.org/src/pkg/text/template/parse/lex.go. Having "grown up" in OO-land, the stateFn pattern was a bit odd to me. However in practical terms this pattern is pretty neat.
- groovy2shoes 14y agoThat's pretty cool. The stateFn pattern seems a lot like recursive descent on a trampoline (iterative descent?).
- regularfry 14y agoThat is interesting, although I can't help wondering if it's overkill here. I'll try it out and see if it makes things simpler.